HCI Bibliography Home | HCI Conferences | IUI Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
IUI Tables of Contents: 050607080910111213-113-214-114-215-115-216-116-2

Proceedings of the 2014 International Conference on Intelligent User Interfaces

Fullname:Proceedings of the 19th International Conference on Intelligent User Interfaces
Editors:Tsvi Kuflik; Oliviero Stock; Joyce Chai; Antonio Krüger
Location:Haifa, Israel
Dates:2014-Feb-24 to 2014-Feb-27
Standard No:ISBN: 978-1-4503-2184-6; ACM DL: Table of Contents; hcibib: IUI14-1
Links:Conference Website
  1. IUI 2014-02-24 Volume 1
    1. Keynote talks
    2. John Riedl session
    3. From touch through air to brain
    4. Learning and skills
    5. Intelligent visual interaction
    6. Users and motion
    7. Leveraging social competencies
    8. Adaptive user interfaces
    9. Posters

IUI 2014-02-24 Volume 1

Keynote talks

Multiadaptive interfaces to cyber-physical environments BIBAFull-Text 1-2
  Wolfgang Wahlster
Networked cyber-physical systems are the basis for intelli-gent environments in a variety of settings such as smart factories, smart transportation systems, smart shops, and smart buildings. However, one of the remaining grand challenges for the Internet of Things is to transform the way how humans interact with and control such cyber-physical environments (CPE). A major goal of our research is the creation of a multiadaptive dialogue management system that is adaptive in multiple ways: adaptive to various CPE, adaptive to diverse modality combinations, adaptive to a variety of interaction metaphors, and adaptive to diverse task domains and user models.
Using augmented reality to create empathic experiences BIBAFull-Text 5-6
  Mark Billinghurst
Intelligent user interfaces have traditionally been used to create systems that respond intelligently to user input. However there is a recent trend towards Empathic Interfaces that are designed to go beyond understanding user input and to recognize emotional state and user feelings. In this presentation we explore how Augmented Reality (AR) can be used to convey that emotional state and so allow users to capture and share emotional experiences. In this way AR not only overlays virtual imagery on the real world, but also can create deeper understanding of user's experience at particular locations and points in time. The recent emergence of truly wearable systems, such as Google Glass, provide a platform for Empathic Communication using AR. Examples will be shown from research conducted at the HIT Lab NZ and other research organizations, and key areas for future research described.

John Riedl session

Who have got answers?: growing the pool of answerers in a smart enterprise social QA system BIBAFull-Text 7-16
  Lin Luo; Fei Wang; Michelle X. Zhou; Yingxin Pan; Hang Chen
On top of an enterprise social platform, we are building a smart social QA system that automatically routes questions to suitable employees who are willing, able, and ready to provide answers. Due to a lack of social QA history (training data) to start with, in this paper, we present an optimization-based approach that recommends both top-matched active (seed) and inactive (prospect) answerers for a given question. Our approach includes three parts. First, it uses a predictive model to find top-ranked seed answerers by their fitness, including their ability and willingness, to answer a question. Second, it uses distance metric learning to discover prospects most similar to the seeds identified in the first step. Third, it uses a constraint-based approach to balance the selection of both seeds and prospects identified in the first two steps. As a result, not only does our solution route questions to top-matched active users, but it also engages inactive users to grow the pool of answerers. Our real-world experiments that routed 114 questions to 684 people identified from 400,000+ employees included 641 prospects (93.7%) and achieved about 70% answering rate with 83% of answers received a lot/full confidence.
Improving business rating predictions using graph based features BIBAFull-Text 17-26
  Amit Tiroshi; Shlomo Berkovsky; Mohamed Ali Kaafar; David Vallet; Terence Chen; Tsvi Kuflik
Many types of recommender systems rely on a rich ensemble of user, item, and context features when generating recommendations for users. The features can be either manually engineered or automatically extracted from the available data, such that feature engineering becomes an important step in the recommendation process. In this work, we propose to leverage graph based representation of the data in order to generate and automatically populate features. We represent the standard user-item rating matrix and some domain metadata, as graph vertices and edges. Then, we apply a suite of graph theory and network analysis metrics to the graph based data representation, to populate features that augment the original user-item ratings data. The augmented data is fed into a classifier that predicts unknown user ratings, which are used for the generation of recommendations. We evaluate the proposed methodology using the recently released Yelp business ratings dataset. Our results indicate that the automatically populated graph features allow for more accurate and robust predictions, with respect to both the variability and sparsity of ratings.
Improving government services with social media feedback BIBAFull-Text 27-36
  Stephen Wan; Cécile Paris
Social media is an invaluable source of feedback not just about consumer products and services but also about the effectiveness of government services. Our aim is to help analysts identify how government services can be improved based on citizen-contributed feedback found in publicly available social media. We present ongoing research for a social media monitoring interactive prototype with federated search and text analysis functionality. The prototype, developed to fit the workflow of social media monitors in the government sector, collects, analyses, and provides overviews of social media content. It facilitates relevance judgements on specific social media posts to decide whether or not to engage online. Our user log analysis validates the original design requirements and indicates ongoing utility to our federated search approach.
Toward crowdsourcing micro-level behavior annotations: the challenges of interface, training, and generalization BIBAFull-Text 37-46
  Sunghyun Park; Philippa Shoemark; Louis-Philippe Morency
Research that involves human behavior analysis usually requires laborious and costly efforts for obtaining micro-level behavior annotations on a large video corpus. With the emerging paradigm of crowdsourcing however, these efforts can be considerably reduced. We first present OCTAB (Online Crowdsourcing Tool for Annotations of Behaviors), a web-based annotation tool that allows precise and convenient behavior annotations in videos, directly portable to popular crowdsourcing platforms. As part of OCTAB, we introduce a training module with specialized visualizations. The training module's design was inspired by an observational study of local experienced coders, and it enables an iterative procedure for effectively training crowd workers online. Finally, we present an extensive set of experiments that evaluates the feasibility of our crowdsourcing approach for obtaining micro-level behavior annotations in videos, showing the reliability improvement in annotation accuracy when properly training online crowd workers. We also show the generalization of our training approach to a new independent video corpus.

From touch through air to brain

Left and right hand distinction for multi-touch tabletop interactions BIBAFull-Text 47-56
  Zhensong Zhang; Fengjun Zhang; Hui Chen; Jiasheng Liu; Hongan Wang; Guozhong Dai
In multi-touch interactive systems, it is of great significance to distinguish which hand of the user is touching the surface in real time. Left-right hand distinction is essential for recognizing the multi-finger gestures and further fully exploring the potential of bimanual interaction. However, left-right hand distinction is beyond the capability of most existing multi-touch systems. In this paper, we present a new method for left and right hand distinction based on the human anatomy, work area, finger orientation and finger position. Considering the ergonomics principles of gesture designing, the body-forearm triangle model was proposed. Furthermore, a heuristic algorithm was introduced to group multi-touch contact points and then made left-right hand distinction. A dataset of 2880 images has been set up to evaluate the proposed left-right hand distinction method. The experimental results demonstrate that our method can guarantee the high recognition accuracy and real time performance in freely bimanual multi-touch interactions.
Improving accuracy in back-of-device multitouch typing: a clustering-based approach to keyboard updating BIBAFull-Text 57-66
  Daniel Buschek; Oliver Schoenleben; Antti Oulasvirta
Recent work has shown that a multitouch sensor attached to the back of a handheld device can allow rapid typing engaging all ten fingers. However, high error rates remain a problem, because the user can not see or feel key-targets on the back. We propose a machine learning approach that can significantly improve accuracy. The method considers hand anatomy and movement ranges of fingers. The key insight is a combination of keyboard and hand models in a hierarchical clustering method. This enables dynamic re-estimation of key-locations while typing to account for changes in hand postures and movement ranges of fingers. We also show that accuracy can be further improved with language models. Results from a user study show improvements of over 40% compared to the previously deployed "naive" approach. We examine entropy as a touch precision metric with respect to typing experience. We also find that the QWERTY layout is not ideal. Finally, we conclude with ideas for further improvements.
User identification using raw sensor data from typing on interactive displays BIBAFull-Text 67-72
  Philipp Mock; Jörg Edelmann; Andreas Schilling; Wolfgang Rosenstiel
Personalized soft-keyboards which adapt to a user's individual typing behavior can reduce typing errors on interactive displays. In multi-user scenarios a personalized model has to be loaded for each participant. In this paper we describe a user identification technique that is based on raw sensor data from an optical touch screen. For classification of users we use a multi-class support vector machine that is trained with grayscale images from the optical sensor. Our implementation can identify a specific user from a set of 12 users with an average accuracy of 97.51% after one keystroke. It can be used to automatically select individual typing models during free-text entry. The resulting authentication process is completely implicit. We furthermore describe how the approach can be extended to automatic loading of personal information and settings.
Teaching motion gestures via recognizer feedback BIBAFull-Text 73-82
  Ankit Kamal; Yang Li; Edward Lank
When using motion gestures, 3D movements of a mobile phone, as an input modality, one significant challenge is how to teach end users the movement parameters necessary to successfully issue a command. Is a simple video or image depicting movement of a smartphone sufficient? Or do we need three-dimensional depictions of movement on external screens to train users? In this paper, we explore mechanisms to teach end users motion gestures, examining two factors. The first factor is how to represent motion gestures: as icons that describe movement, video that depicts movement using the smartphone screen, or a Kinect-based teaching mechanism that captures and depicts the gesture on an external display in three-dimensional space. The second factor we explore is recognizer feedback, i.e. a simple representation of the proximity of a motion gesture to the desired motion gesture based on a distance metric extracted from the recognizer. We show that, by combining video with recognizer feedback, participants master motion gestures equally quickly as end users that learn using a Kinect. These results demonstrate the viability of training end users to perform motion gestures using only the smartphone display.
A brain-computer interface for high-level remote control of an autonomous, reinforcement-learning-based robotic system for reaching and grasping BIBAFull-Text 83-88
  Thomas Lampe; Lukas D. J. Fiederer; Martin Voelker; Alexander Knorr; Martin Riedmiller; Tonio Ball
We present an Internet-based brain-computer interface (BCI) for controlling an intelligent robotic device with autonomous reinforcement-learning. BCI control was achieved through dry-electrode electroencephalography (EEG) obtained during imaginary movements. Rather than using low-level direct motor control, we employed a high-level control scheme of the robot, acquired via reinforcement learning, to keep the users cognitive load low while allowing control a reaching-grasping task with multiple degrees of freedom. High-level commands were obtained by classification of EEG responses using an artificial neural network approach utilizing time-frequency features and conveyed through an intuitive user interface. The novel combination of a rapidly operational dry electrode setup, autonomous control and Internet connectivity made it possible to conveniently interface subjects in an EEG laboratory with remote robotic devices in a closed-loop setup with online visual feedback of the robots actions to the subject. The same approach is also suitable to provide home-bound patients with the possibility to control state-of-the-art robotic devices currently confined to a research environment. Thereby, our BCI approach could help severely paralyzed patients by facilitating patient-centered research of new means of communication, mobility and independence.
Mimetic interaction spaces: controlling distant displays in pervasive environments BIBAFull-Text 89-94
  Hanae Rateau; Laurent Grisoni; Bruno De Araujo
Pervasive computing is a vision that has been an inspiring long-term target for many years now. Interaction techniques that allow one user to efficiently control many screens, or that allow several users to collaborate on one distant screen, are still hot topics, and are often considered as two different questions. Standard approaches require a strong coupling between the physical location of input device, and users. We propose to consider these two questions through the same basic concept, that uncouples physical location and user input, using a mid-air approach. We present the concept of mimetic interaction spaces (MIS), a dynamic user-definition of an imaginary input space thanks to an iconic gesture, that can be used to define mid-air interaction techniques. We describe a participative design user-study, that shows this technique has interesting acceptability and elicit some definition and deletion gestures. We finally describe a design space for MIS-based interaction, and show how such concept may be used for multi-screen control, as well as screen sharing in pervasive environments.

Learning and skills

Recognition of understanding level and language skill using measurements of reading behavior BIBAFull-Text 95-104
  Pascual Martínez-Gómez; Akiko Aizawa
The reading act is an intimate and elusive process that is important to understand. Psycholinguists have long studied the effects of task, personal or document characteristics on reading behavior. An essential factor in the success of those studies lies in the capability of analyzing eye-movements. These studies aim to recognize causal effects on patterns of eye-movements, by contriving variations in task, personal or document characteristics. In this work, we follow the opposite direction. We present a formal framework to recognize reader's level of understanding and language skill given measurements of reading behavior via eye-gaze data. We show significant error reductions to recognize these attributes and provide a detailed study of the most discriminative features.
Towards facilitating user skill acquisition: identifying untrained visualization users through eye tracking BIBAFull-Text 105-114
  Dereck Toker; Ben Steichen; Matthew Gingerich; Cristina Conati; Giuseppe Carenini
A key challenge for information visualization designers lies in developing systems that best support users in terms of their individual abilities, needs, and preferences. However, most visualizations require users to first gather a certain set of skills before they can efficiently process the displayed information. This paper presents a first step towards designing visualizations that provide personalized support in order to ease the so-called 'learning curve' during a user's skill acquisition phase. We present prediction models, trained on users' gaze data, that can identify if users are still in the skill acquisition phase or if they have gained the necessary abilities. The paper first reveals that users exhibit the learning curve even during the usage of simple information visualizations, and then shows that we can generate reasonably accurate predictions about a user's skill acquisition using solely their eye gaze behavior.
Active learning of intuitive control knobs for synthesizers using gaussian processes BIBAFull-Text 115-124
  Cheng-Zhi Anna Huang; David Duvenaud; Kenneth C. Arnold; Brenton Partridge; Josiah W. Oberholtzer; Krzysztof Z. Gajos
Typical synthesizers only provide controls to the low-level parameters of sound-synthesis, such as wave-shapes or filter envelopes. In contrast, composers often want to adjust and express higher-level qualities, such as how "scary" or "steady" sounds are perceived to be. We develop a system which allows users to directly control abstract, high-level qualities of sounds. To do this, our system learns functions that map from synthesizer control settings to perceived levels of high-level qualities. Given these functions, our system can generate high-level knobs that directly adjust sounds to have more or less of those qualities. We model the functions mapping from control-parameters to the degree of each high-level quality using Gaussian processes, a nonparametric Bayesian model. These models can adjust to the complexity of the function being learned, account for nonlinear interaction between control-parameters, and allow us to characterize the uncertainty about the functions being learned. By tracking uncertainty about the functions being learned, we can use active learning to quickly calibrate the tool, by querying the user about the sounds the system expects to most improve its performance. We show through simulations that this model-based active learning approach learns high-level knobs on certain classes of target concepts faster than several baselines, and give examples of the resulting automatically-constructed knobs which adjust levels of non-linear, high-level concepts.
Visualizing expert solutions in exploratory learning environments using plan recognition BIBAFull-Text 125-132
  Or Seri; Kobi Gal
Exploratory Learning Environments (ELE) are open-ended and flexible software, supporting interaction styles that include exogenous actions and trial-and-error. This paper shows that using AI techniques to visualize worked examples in ELEs improves students' generalization of mathematical concepts across problems, as measured by their performance. Students were exposed to a worked example of a problem solution using an ELE for statistics education. One group in the study was presented with a hierarchical plan of relevant activities that emphasized the sub-goals and the structure relating to the solution. This visualization used an AI algorithm to match a log of activities in the ELEs to ideal solutions. We measured students' performance when using the ELE to solve new problems that required generalization of concepts introduced in the example solution. The results showed that students who were shown the plan visualization significantly outperformed other students who were presented with a step-by-step list of actions in the software used to generate the same solution to the example problem. Analysis of students' explanations of the problem solution shows that the students in the former condition also demonstrated deeper understanding of the solution process. These results demonstrate the benefit to students when using AI technology to visualize worked examples in ELEs and suggests future applications of this approach to actively support students' learning and teachers' understanding of students' activities.
Expediting expertise: supporting informal social learning in the enterprise BIBAFull-Text 133-142
  Jennifer Lai; Jie Lu; Shimei Pan; Danny Soroker; Mercan Topkara; Justin Weisz; Jeff Boston; Jason Crawford
In this paper, we present Expediting Expertise, a system designed to provide structured support to the otherwise informal process of social learning in the enterprise. It employs a data-driven approach where online content is automatically analyzed and categorized into relevant topics, topic-specific user expertise is calculated by comparing the models of individual users against those of the experts, and personalized recommendation of learning activities is created accordingly to facilitate expertise development. The system's UI is designed to provide users with ongoing feedback of current expertise, progress, and comparison with others. Learning recommendation is visualized with an interactive treemap which presents estimated return on investment and distance to current expertise for each recommended learning activity. Evaluation of the system showed very positive results.

Intelligent visual interaction

SpiderEyes: designing attention- and proximity-aware collaborative interfaces for wall-sized displays BIBAFull-Text 143-152
  Jakub Dostal; Uta Hinrichs; Per Ola Kristensson; Aaron Quigley
With the proliferation of large multi-faceted datasets, a critical question is how to design collaborative environments, in which this data can be analysed in an efficient and insightful manner. Exploiting people's movements and distance to the data display and to collaborators, proxemic interactions can potentially support such scenarios in a fluid and seamless way, supporting both tightly coupled collaboration as well as parallel explorations. In this paper we introduce the concept of collaborative proxemics: enabling groups of people to collaboratively use attention- and proximity-aware applications. To help designers create such applications we have developed SpiderEyes: a system and toolkit for designing attention- and proximity-aware collaborative interfaces for wall-sized displays. SpiderEyes is based on low-cost technology and allows accurate markerless attention-aware tracking of multiple people interacting in front of a display in real-time. We discuss how this toolkit can be applied to design attention- and proximity-aware collaborative scenarios around large wall-sized displays, and how the information visualisation pipeline can be extended to incorporate proxemic interactions.
Frequence: interactive mining and visualization of temporal frequent event sequences BIBAFull-Text 153-162
  Adam Perer; Fei Wang
Extracting insights from temporal event sequences is an important challenge. In particular, mining frequent patterns from event sequences is a desired capability for many domains. However, most techniques for mining frequent patterns are ineffective for real-world data that may be low-resolution, concurrent, or feature many types of events, or the algorithms may produce results too complex to interpret. To address these challenges, we propose Frequence, an intelligent user interface that integrates data mining and visualization in an interactive hierarchical information exploration system for finding frequent patterns from longitudinal event sequences. Frequence features a novel frequent sequence mining algorithm to handle multiple levels-of-detail, temporal context, concurrency, and outcome analysis. Frequence also features a visual interface designed to support insights, and support exploration of patterns of the level-of-detail relevant to users. Frequence's effectiveness is demonstrated with two use cases: medical research mining event sequences from clinical records to understand the progression of a disease, and social network research using frequent sequences from Foursquare to understand the mobility of people in an urban environment.
Time-offset interaction with a holocaust survivor BIBAFull-Text 163-168
  Ron Artstein; David Traum; Oleg Alexander; Anton Leuski; Andrew Jones; Kallirroi Georgila; Paul Debevec; William Swartout; Heather Maio; Stephen Smith
Time-offset interaction is a new technology that allows for two-way communication with a person who is not available for conversation in real time: a large set of statements are prepared in advance, and users access these statements through natural conversation that mimics face-to-face interaction. Conversational reactions to user questions are retrieved through a statistical classifier, using technology that is similar to previous interactive systems with synthetic characters; however, all of the retrieved utterances are genuine statements by a real person. Recordings of answers, listening and idle behaviors, and blending techniques are used to create a persistent visual image of the person throughout the interaction. A proof-of-concept has been implemented using the likeness of Pinchas Gutter, a Holocaust survivor, enabling short conversations about his family, his religious views, and resistance. This proof-of-concept has been shown to dozens of people, from school children to Holocaust scholars, with many commenting on the impact of the experience and potential for this kind of interface.

Users and motion

Hierarchical route maps for efficient navigation BIBAFull-Text 169-178
  Fangzhou Wang; Yang Li; Daisuke Sakamoto; Takeo Igarashi
One of the difficulties with standard route maps is accessing to multi-scale routing information. The user needs to display maps in both a large scale to see details and a small scale to see an overview, but this requires tedious interaction such as zooming in and out. We propose to use a hierarchical structure for a route map, called a "Route Tree", to address this problem, and describe an algorithm to automatically construct such a structure. A Route Tree is a hierarchical grouping of all small route segments to allow quick access to meaningful large and small-scale views. We propose two Route Tree applications, "RouteZoom" for interactive map browsing and "TreePrint" for route information printing, to show the applicability and usability of the structure. We conducted a preliminary user study on RouteZoom, and the results showed that RouteZoom significantly lowers the interaction cost for obtaining information from a map compared to a traditional interactive map.
STAR-CITY: semantic traffic analytics and reasoning for CITY BIBAFull-Text 179-188
  Freddy Lécué; Simone Tallevi-Diotallevi; Jer Hayes; Robert Tucker; Veli Bicer; Marco Luca Sbodio; Pierpaolo Tommasi
This paper presents STAR-CITY, a system supporting semantic traffic analytics and reasoning for city. STAR-CITY, which integrates (human and machine-based) sensor data using variety of formats, velocities and volumes, has been designed to provide insight on historical and real-time traffic conditions, all supporting efficient urban planning. Our system demonstrates how the severity of road traffic congestion can be smoothly analyzed, diagnosed, explored and predicted using semantic web technologies. We present how semantic diagnosis and predictive reasoning, both using and interpreting semantics of data to deliver useful, accurate and consistent inferences, have been exploited and adapted systematized in an intelligent user interface. Our prototype of semantics-aware traffic analytics and reasoning, experimented in Dublin City Ireland, works and scales efficiently with historical together with real live and heterogeneous stream data.
Location sharing privacy preference: analysis and personalized recommendation BIBAFull-Text 189-198
  Jierui Xie; Bart Piet Knijnenburg; Hongxia Jin
Location-based systems are becoming more popular with the explosive growth in popularity of smart phones. However, the user adoption of these systems is hindered by growing user concerns about privacy. To design better location-based systems that attract more user adoption and protect users from information under/overexposure, it is highly desirable to understand users' location sharing and privacy preferences. This paper makes two main contributions. First, by studying users' location sharing privacy preferences with three groups of people (i.e., Family, Friend and Colleague) in different contexts, including check-in time, companion and emotion, we reveal that location sharing behaviors are highly dynamic, context-aware, audience-aware and personal. In particular, we find that emotion and companion are good contextual predictors of privacy preferences. Moreover, we find that there are strong similarities or correlations among contexts and groups. Our second contribution is to show, in light of the user study, that despite the dynamic and context-dependent nature of location sharing, it is still possible to predict a user's in-situ sharing preference in various contexts. More specifically, we explore whether it is possible to give users a personalized recommendation of the sharing setting they are most likely to prefer, based on context similarity, group correlation and collective check-in preference. PPRec, the proposed recommendation algorithm that incorporates the above three elements, delivers personalized recommendations that could be helpful to reduce both user's burden and privacy risk. It also provides additional insights into the relative usefulness of different personal and contextual factors in predicting users' sharing behavior.
Audio haptic videogaming for developing wayfinding skills in learners who are blind BIBAFull-Text 199-208
  Jaime Sánchez; Marcia de Borba Campos; Matías Espinoza; Lotfi B. Merabet
Interactive digital technologies are currently being developed as a novel tool for education and skill development. Audiopolis is an audio and haptic based videogame designed for developing orientation and mobility (O&M) skills in people who are blind. We have evaluated the cognitive impact of videogame play on O&M skills by assessing performance on a series of behavioral tasks carried out in both indoor and outdoor virtual spaces. Our results demonstrate that the use of Audiopolis had a positive impact on the development and use of O&M skills in school-aged learners who are blind. The impact of audio and haptic information on learning is also discussed.
Triangle charades: a data-collection game for recognizing actions in motion trajectories BIBAFull-Text 209-214
  Melissa Roemmele; Haley Archer-McClellan; Andrew S. Gordon
Humans have a remarkable tendency to anthropomorphize moving objects, ascribing to them intentions and emotions as if they were human. Early social psychology research demonstrated that animated film clips depicting the movements of simple geometric shapes could elicit rich interpretations of intentional behavior from viewers. In attempting to model this reasoning process in software, we first address the problem of automatically recognizing humanlike actions in the trajectories of moving shapes. There are two main difficulties. First, there is no defined vocabulary of actions that are recognizable to people from motion trajectories. Second, in order for an automated system to learn actions from motion trajectories using machine-learning techniques, a vast amount of these action-trajectory pairs is needed as training data. This paper describes an approach to data collection that resolves both of these problems. In a web-based game, called Triangle Charades, players create motion trajectories for actions by animating a triangle to depict those actions. Other players view these animations and guess the action they depict. An action is considered recognizable if players can correctly guess it from animations. To move towards defining a controlled vocabulary and collecting a large dataset, we conducted a pilot study in which 87 users played Triangle Charades. Based on this data, we computed a simple metric for action recognizability. Scores on this metric formed a gradual linear pattern, suggesting there is no clear cutoff for determining if an action is recognizable from motion data. These initial results demonstrate the advantages of using a game to collect data for this action recognition task.
Comparison of speech-based in-car HMI concepts in a driving simulation study BIBAFull-Text 215-224
  Hansjörg Hofmann; Vanessa Tobisch; Ute Ehrlich; André Berton; Angela Mahr
This paper reports experimental results from a driving simulation study in order to compare different speech-based in-car human-machine interface concepts. The effects of the use of a command-based and a conversational in-car speech dialog system on usability and driver distraction are evaluated. Different graphical user interface concepts have been designed in order to investigate their potential supportive or distracting effects. The results show that only few differences concerning speech dialog quality were found when comparing the speech dialog strategies. The command-based dialog was slightly better accepted than the conversational dialog, which can be attributed to the limited performance of the system's language understanding component. No differences in driver distraction were revealed. Moreover, the study revealed that speech dialog systems without graphical user interface were accepted by participants in the driving environment and that the use of a graphical user interface impaired the driving performance and increased gaze-based distraction. In the driving scenario, the choice of speech dialog strategies does not have a strong influence on usability and no influence on driver distraction. Instead, when designing the graphical user interface of an in-car speech dialog systems, developers should consider reducing the content presented on the display device in order to reduce driver distraction.

Leveraging social competencies

Overt or subtle?: supporting group conversations with automatically targeted directives BIBAFull-Text 225-234
  Gianluca Schiavo; Alessandro Cappelletti; Eleonora Mencarini; Oliviero Stock; Massimo Zancanaro
In this paper, we present a system that acts as an automatic facilitator by supporting the flow of communication in a group conversation activity. The system monitors the group members' non-verbal behavior and promotes balanced participation, giving targeted directives to the participants through peripheral displays. We describe an initial study to compare two ways of influencing participants' social dynamics: overt directives, explicit recommendations of social actions displayed in the form of text; or subtle directives, where the same recommendations are provided in an implicit manner. Our study indicates that, when the participants understand how the implicit messages work, the subtle facilitation is regarded as more useful than the overt one and it is considered to more positively influence the group behavior.
See what you want to see: visual user-driven approach for hybrid recommendation BIBAFull-Text 235-240
  Denis Parra; Peter Brusilovsky; Christoph Trattner
Research in recommender systems has traditionally focused on improving the predictive accuracy of recommendations by developing new algorithms or by incorporating new sources of data. However, several studies have shown that accuracy does not always correlate with a better user experience, leading to recent research that puts emphasis on Human-Computer Interaction in order to investigate aspects of the interface and user characteristics that influence the user experience on recommender systems. Following this new research this paper presents SetFusion, a visual user-controllable interface for hybrid recommender system. Our approach enables users to manually fuse and control the importance of recommender strategies and to inspect the fusion results using an interactive Venn diagram visualization. We analyze the results of two field studies in the context of a conference talk recommendation system, performed to investigate the effect of user controllability in a hybrid recommender. Behavioral analysis and subjective evaluation indicate that the proposed controllable interface had a positive effect on the user experience.
Employing linked data and dialogue for modelling cultural awareness of a user BIBAFull-Text 241-246
  Ronald Denaux; Vania Dimitrova; Lydia Lau; Paul Brna; Dhaval Thakker; Christina Steiner
Intercultural competence is an essential 21st Century skill. A key issue for developers of cross-cultural training simulators is the need to provide relevant learning experience adapted to the learner's abilities. This paper presents a dialogic approach for a quick assessment of the depth of a learner's current intercultural awareness as part of the EU ImREAL project. To support the dialogue, Linked Data is seen as a rich knowledge base for a diverse range of resources on cultural aspects. This paper investigates how semantic technologies could be used to: (a) extract a pool of concrete culturally-relevant facts from DBpedia that can be linked to various cultural groups and to the learner, (b) model a learner's knowledge on a selected set of cultural themes and (c) provide a novel, adaptive and user-friendly, user modelling dialogue for cultural awareness. The usability and usefulness of the approach is evaluated by CrowdFlower and Expert Inspection.
Who will retweet this? BIBAFull-Text 247-256
  Kyumin Lee; Jalal Mahmud; Jilin Chen; Michelle Zhou; Jeffrey Nichols
There has been much effort on studying how social media sites, such as Twitter, help propagate information in different situations, including spreading alerts and SOS messages in an emergency. However, existing work has not addressed how to actively identify and engage the right strangers at the right time on social media to help effectively propagate intended information within a desired time frame. To ad-dress this problem, we have developed two models: (i) a feature-based model that leverages peoples' exhibited social behavior, including the content of their tweets and social interactions, to characterize their willingness and readiness to propagate information on Twitter via the act of retweeting; and (ii) a wait-time model based on a user's previous retweeting wait times to predict her next retweeting time when asked. Based on these two models, we build a recommender system that predicts the likelihood of a stranger to retweet information when asked, within a specific time window, and recommends the top-N qualified strangers to engage with. Our experiments, including live studies in the real world, demonstrate the effectiveness of our work.

Adaptive user interfaces

Tagging-by-search: automatic image region labeling using gaze information obtained from image search BIBAFull-Text 257-266
  Tina Walber; Chantal Neuhaus; Ansgar Scherp
Labeled image regions provide very valuable information that can be used in different settings such as image search. The manual creation of region labels is a tedious task. Fully automatic approaches lack understanding the image content sufficiently due to the huge variety of depicted objects. Our approach benefits from the expected spread of eye tracking hardware and uses gaze information obtained from users performing image search tasks to automatically label image regions. This allows to exploit the human capabilities regarding the visual perception of image content while performing daily routine tasks. In an experiment with 23 participants, we show that it is possible to assign search terms to photo regions by means of gaze analysis with an average precision of 0.56 and an average F-measure of 0.38 over 361 photos. The participants performed different search tasks while their gaze was recorded. The results of the experiment show that the gaze-based approach performs significantly better than a baseline approach based on saliency maps.
Using eye-tracking to support interaction with layered 3D interfaces on stereoscopic displays BIBAFull-Text 267-272
  Florian Alt; Stefan Schneegass; Jonas Auda; Rufat Rzayev; Nora Broy
In this paper, we investigate the concept of gaze-based interaction with 3D user interfaces. We currently see stereo vision displays becoming ubiquitous, particularly as auto-stereoscopy enables the perception of 3D content without the use of glasses. As a result, application areas for 3D beyond entertainment in cinema or at home emerge, including work settings, mobile phones, public displays, and cars. At the same time, eye tracking is hitting the consumer market with low-cost devices. We envision eye trackers in the future to be integrated with consumer devices (laptops, mobile phones, displays), hence allowing the user's gaze to be analyzed and used as input for interactive applications. A particular challenge when applying this concept to 3D displays is that current eye trackers provide the gaze point in 2D only (x and y coordinates). In this paper, we compare the performance of two methods that use the eye's physiology for calculating the gaze point in 3D space, hence enabling gaze-based interaction with stereoscopic content. Furthermore, we provide a comparison of gaze interaction in 2D and 3D with regard to user experience and performance. Our results show that with current technology, eye tracking on stereoscopic displays is possible with similar performance as on standard 2D screens.
On user behaviour adaptation under interface change BIBAFull-Text 273-278
  Benjamin Rosman; Subramanian Ramamoorthy; M. M. Hassan Mahmud; Pushmeet Kohli
Different interfaces allow a user to achieve the same end goal through different action sequences, e.g., command lines vs. drop down menus. Interface efficiency can be described in terms of a cost incurred, e.g., time taken, by the user in typical tasks. Realistic users arrive at evaluations of efficiency, hence making choices about which interface to use, over time, based on trial and error experience. Their choices are also determined by prior experience, which determines how much learning time is required. These factors have substantial effect on the adoption of new interfaces. In this paper, we aim at understanding how users adapt under interface change, how much time it takes them to learn to interact optimally with an interface, and how this learning could be expedited through intermediate interfaces. We present results from a series of experiments that make four main points: (a) different interfaces for accomplishing the same task can elicit significant variability in performance, (b) switching interfaces can result in adverse sharp shifts in performance, (c) subject to some variability, there are individual thresholds on tolerance to this kind of performance degradation with an interface, causing users to potentially abandon what may be a pretty good interface, and (d) our main result -- shaping user learning through the presentation of intermediate interfaces can mitigate the adverse shifts in performance while still enabling the eventual improved performance with the complex interface upon the user becoming suitably accustomed. In our experiments, human users use keyboard based interfaces to navigate a simulated ball through a maze. Our results are a first step towards interface adaptation algorithms that architect choice to accommodate personality traits of realistic users.
Natural language queries over heterogeneous linked data graphs: a distributional-compositional semantics approach BIBAFull-Text 279-288
  Andre Freitas; Edward Curry
The demand to access large amounts of heterogeneous structured data is emerging as a trend for many users and applications. However, the effort involved in querying heterogeneous and distributed third-party databases can create major barriers for data consumers. At the core of this problem is the semantic gap between the way users express their information needs and the representation of the data. This work aims to provide a natural language interface and an associated semantic index to support an increased level of vocabulary independency for queries over Linked Data/Semantic Web datasets, using a distributional-compositional semantics approach. Distributional semantics focuses on the automatic construction of a semantic model based on the statistical distribution of co-occurring words in large-scale texts. The proposed query model targets the following features: (i) a principled semantic approximation approach with low adaptation effort (independent from manually created resources such as ontologies, thesauri or dictionaries), (ii) comprehensive semantic matching supported by the inclusion of large volumes of distributional (unstructured) commonsense knowledge into the semantic approximation process and (iii) expressive natural language queries. The approach is evaluated using natural language queries on an open domain dataset and achieved avg. recall=0.81, mean avg. precision=0.62 and mean reciprocal rank=0.49.
MILA -- S: generation of agent-based simulations from conceptual models of complex systems BIBAFull-Text 289-298
  David A. Joyner; Ashok K. Goel; Nicolas M. Papin
Scientists use both conceptual models and executable simulations to help them make sense of the world. Models and simulations each have unique affordances and limitations, and it is useful to leverage their affordances to mitigate their respective limitations. One way to do this is by generating the simulations based on the conceptual models, preserving the capacity for rapid revision and knowledge sharing allowed by the conceptual models while extending them to provide the repeated testing and feedback of the simulations. In this paper, we present an interactive system called MILA?S for generating agent-based simulations from conceptual models of ecological systems. Designed with STEM education in mind, this user-centered interface design allows the user to construct a Component-Mechanism-Phenomenon conceptual model of a complex system, and then compile the conceptual model into an executable NetLogo simulation. In this paper, we present the results of a pilot study with this interface with about 50 middle school students in the context of learning about ecosystems.
Adaptive click-and-cross: adapting to both abilities and task improves performance of users with impaired dexterity BIBAFull-Text 299-304
  Louis Li; Krzysztof Z. Gajos
Computer users with impaired dexterity often have difficulty accessing small, densely packed user interface elements. Past research in software-based solutions has mainly employed two approaches: modifying the interface and modifying the interaction with the cursor. Each approach, however, has limitations. Modifying the user interface by enlarging interactive elements makes access efficient for simple interfaces but increases the cost of navigation for complex ones by displacing items to screens that require tabs or scrolling to reach. Modifying the interaction with the cursor makes access possible to unmodified interfaces but may perform poorly on densely packed targets or require the user to perform multiple steps. We developed a new approach that combines the strengths of the existing approaches while minimizing their shortcomings, introducing only minimal distortion to the original interface while making access to frequently used parts of the user interface efficient and access to all other parts possible. We instantiated this concept as Adaptive Click-and-Cross, a novel interaction technique. Our user study demonstrates that, for sufficiently complex interfaces, Adaptive Click-and-Cross slightly improves the performance of users with impaired dexterity compared to only modifying the interface or only modifying the cursor.


Towards a user experience design framework for adaptive spoken dialogue in automotive contexts BIBAFull-Text 305-310
  Pontus Wärnestål; Fredrik Kronlid
We present an initial set of design principles for designing efficient, effective, coherent, and desirable adaptive spoken interaction for traffic information and navigation. The principles are based on a qualitative analysis of driver interactions with an adaptive speech prototype along with driver interviews. The derived set of principles range from high-level fundamental design values, conceptual and behavioral principles, to low-level interface-level principles that can guide the design of adaptive spoken dialogue interaction in the car from a user experience perspective.
A practical framework for constructing structured drawings BIBAFull-Text 311-316
  Salman Cheema; Sarah Buchanan; Sumit Gulwani; Joseph J., Jr. LaViola
We describe a novel theoretical framework for modeling structured drawings which contain one or more patterns of repetition in their constituent elements. We then present PatternSketch, a sketch-based drawing tool built using our framework to allow quick construction of structured drawings. PatternSketch can recognize and beautify drawings containing line segments, polylines, arcs, and circles. Users can employ a series of gestures to identify repetitive elements and create new elements based on automatically inferred patterns. PatternSketch leverages the programming-by-example (PBE) paradigm, enabling it to infer non-trivial patterns from a few examples. We show that PatternSketch, with its sketch-based user interface and a unique pattern inference algorithm, enables efficient and natural construction of structured drawings.
Minimizing user effort in transforming data by example BIBAFull-Text 317-322
  Bo Wu; Pedro Szekely; Craig A. Knoblock
Programming by example enables users to transform data formats without coding. To be practical, the method must synthesize the correct transformation with minimal user input. We present a method that minimizes user effort by color-coding the transformation result and recommending specific records where the user should provide examples. Simulation results and a user study show that our method significantly reduces user effort and increases the success rate for synthesizing correct transformation programs by example.
Exploring head tracked head mounted displays for first person robot teleoperation BIBAFull-Text 323-328
  Corey Pittman; Joseph J., Jr. LaViola
We explore the capabilities of head tracking combined with head mounted displays (HMD) as an input modality for robot navigation. We use a Parrot AR Drone to test five techniques which include metaphors for plane-like banking control, car-like turning control and virtual reality-inspired translation and rotation schemes which we compare with a more traditional game controller interface. We conducted a user study to observe the effectiveness of each of the interfaces we developed in navigating through a number of archways in an indoor course. We examine a number of qualitative and quantitative metrics to determine performance and preference among each metaphor. Our results show an appreciation for head rotation based controls over other head gesture techniques, with the classic controller being preferred overall. We discuss possible shortcomings with head tracked HMDs as a primary input method as well as propose improved metaphors that alleviate some of these drawbacks.
A mixed reality head-mounted text translation system using eye gaze input BIBAFull-Text 329-334
  Takumi Toyama; Daniel Sonntag; Andreas Dengel; Takahiro Matsuda; Masakazu Iwamura; Koichi Kise
Efficient text recognition has recently been a challenge for augmented reality systems. In this paper, we propose a system with the ability to provide translations to the user in real-time. We use eye gaze for more intuitive and efficient input for ubiquitous text reading and translation in head mounted displays (HMDs). The eyes can be used to indicate regions of interest in text documents and activate optical-character-recognition (OCR) and translation functions. Visual feedback and navigation help in the interaction process, and text snippets with translations from Japanese to English text snippets, are presented in a see-through HMD. We focus on travelers who go to Japan and need to read signs and propose two different gaze gestures for activating the OCR text reading and translation function. We evaluate which type of gesture suits our OCR scenario best. We also show that our gaze-based OCR method on the extracted gaze regions provide faster access times to information than traditional OCR approaches. Other benefits include that visual feedback of the extracted text region can be given in real-time, the Japanese to English translation can be presented in real-time, and the augmentation of the synchronized and calibrated HMD in this mixed reality application are presented at exact locations in the augmented user view to allow for dynamic text translation management in head-up display systems.
AllAboard: visual exploration of cellphone mobility data to optimise public transport BIBAFull-Text 335-340
  Giusy di lorenzo; Marco Luca Sbodio; Francesco Calabrese; Michele Berlingerio; Rahul Nair; Fabio Pinelli
The deep penetration of mobile phones offers cities the ability to opportunistically monitor citizens' mobility and use data-driven insights to better plan and manage services. In this context, transit operators can leverage pervasive mobile sensing to better match observed demand for travel with their service offerings. In this paper we present AllAboard, an intelligent tool that analyses cellphone data to helps city authorities in exploring urban mobility and optimizing public transport. An interactive user interface allows transit operators to explore the travel demand in both space and time, evaluate the quality of service that a transit network provides to the citizens, and test scenarios for transit network improvements. The system has been tested using real telecommunication data for the city of Abidjan, Ivory Coast, and evaluated from a data mining, optimization and user prospective.
Exploring customer specific KPI selection strategies for an adaptive time critical user interface BIBAFull-Text 341-346
  Ingo R. Keck; Robert J. Ross
Rapid growth in the number of measures available to describe customer-organization relationships is presenting a serious challenge for Business Intelligence (BI) interface developers as they attempt to provide business users with key customer information without requiring users to painstakingly sift through many interface windows and layers. In this paper we introduce a prototype Intelligent User Interface that we have deployed to partially address this issue. The interface builds on machine learning techniques to construct a ranking model of Key Performance Indicators (KPIs) that are used to select and present the most important customer metrics that can be made available to business users in time critical environments. We provide an overview of the prototype application, the underlying models used for KPI selection, and a comparative evaluation of machine learning and closed form solutions to the ranking and selection problems. Results show that the machine learning based method outperformed the closed form solution with a 66.5% accuracy rate on multi-label attribution in comparison to 54.1% for the closed form solution.
Improving cross-domain information sharing in care coordination using semantic web technologies BIBAFull-Text 347-352
  Spyros Kotoulas; Vanessa Lopez; Marco Luca Sbodio; Pierpaolo Tommasi; Martin Stephenson; Pol Mac Aonghusa
We present an approach to access and consolidate complex information spanning multiple specialist domains and make it available to non-experts. We are using a combination of business rules and contextual exploration to reduce interface complexity and improve consumability. We present a use case and a prototype on top of a real-world enterprise solution for coordinating Social care and Health care. We evaluate our system through a user study. Our results indicate that our approach reduces the time required to obtain business results compared to a baseline graph exploration approach.
AR Lamp: interactions on projection-based augmented reality for interactive learning BIBAFull-Text 353-358
  Jeongyun Kim; Jonghoon Seo; Tack-Don Han
Today, people use a computer almost everywhere. At the same time, they still do their work in the old-fashioned way, such as using a pen and paper. A pen is often used in many fields because it is easy to use and familiar. On the other hand, however, it is a quite inconvenient because the information printed on paper is static. If digital features are added to this paper environment, the users can do their work more easily and efficiently. AR (augmented reality) Lamp is a stand-type projector and camera embedded system with the form factor of a desk lamp. Its users can modify the virtually augmented content on top of the paper with seamlessly combined virtual and physical worlds. AR is quite appealing, but it is difficult to popularize due to the lack of interaction. In this paper, the interaction methods that people can use easily and intuitively are focused on. A high-fidelity prototype of the system is presented, and a set of novel interactions is demonstrated. A pilot evaluation of the system is also reported to explore its usage possibility.
Steptorials: mixed-initiative learning of high-functionality applications BIBAFull-Text 359-364
  Henry Lieberman; Elizabeth Rosenzweig; Christopher Fry
How can a new user learn an unfamiliar application, especially if it is a high-functionality (hi-fun) application, like Photoshop, Excel, or programming language IDE. Many applications provide introductory videos, illustrative examples, and documentation on individual operations. Tests show, however, that novice users are likely to ignore the provided help, and try to learn by exploring the application first. In a hi-fun application, though, the user may lack understanding of the basic concepts of an application's operation, even though they were likely explained in the (ignored) documentation. This paper introduces steptorials ("stepper tutorials"), a new interaction strategy for learning hi-fun applications. A steptorial aims to teach the user how to work through a simple, but nontrivial, example of using the application. Steptorials are unique because they allow varying the autonomy of the user at every step. A steptorial has a control structure of a reversible programming language stepper. The user may choose, at any time, to be shown how to do a step, be guided through it, use the application interface without constraint, or to return to a previous step. It reduces the risk in either trying new operations yourself, or conversely, the risk of ceding control to the computer. It introduces a new paradigm of mixed-initiative learning of application interfaces.
MIXPLORATION: rethinking the audio mixer interface BIBAFull-Text 365-370
  Mark Cartwright; Bryan Pardo; Josh Reiss
A typical audio mixer interface consists of faders and knobs that control the amplitude level as well as processing (e.g. equalization, compression and reverberation) parameters of individual tracks. This interface, while widely used and effective for optimizing a mix, may not be the best interface to facilitate exploration of different mixing options. In this work, we rethink the mixer interface, describing an alternative interface for exploring the space of possible mixes of four audio tracks. In a user study with 24 participants, we compared the effectiveness of this interface to the traditional paradigm for exploring alternative mixes. In the study, users responded that the proposed alternative interface facilitated exploration and that they considered the process of rating mixes to be beneficial.