HCI Bibliography Home | HCI Conferences | UIST Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
UIST Tables of Contents: 091011-111-212-112-213-113-214-114-215-115-2

Proceedings of the 2015 ACM Symposium on User Interface Software and Technology

Fullname:UIST'15: Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology
Editors:Celine Latulipe; Bjoern Hartmann; Tovi Grossman
Location:Charlotte, North Carolina
Dates:2015-Nov-08 to 2015-Nov-11
Standard No:ISBN: 978-1-4503-3779-3; ACM DL: Table of Contents; hcibib: UIST15-1
Links:Conference Website
  1. UIST 2015-11-05 Volume 1
    1. Opening Keynote Address
    2. Session 1A: Tactile Feedback
    3. Session 1B: Large Displays, Large Movements
    4. Session 2A: Fabrication 1 -- Augmentation
    5. Session 2B: 3D & Augmented Reality
    6. Session 3A: Sensing Techniques
    7. Session 3B: Intelligent Information Interfaces
    8. Session 4A: Fabrication 2 -- Flexible and Printed Electronics
    9. Session 4B: Tools for Programmers
    10. Session 5A: Touch Input
    11. Session 5B: Tangibles
    12. Session 6A: Gaze
    13. Session 6B: Pushing Virtual and Physical Envelopes
    14. Session 7A: Wearable and Mobile Interactions
    15. Session 7B: Neurons, Affect, Ambiguity
    16. Session 8A: Hands and Fingers
    17. Session 8B: Fabrication 3 -- Complex Shapes and Properties
    18. Session 9A: Online Education
    19. Session 9B: Pens, Mice and Sensor Strips
    20. Closing Keynote Address

UIST 2015-11-05 Volume 1

Opening Keynote Address

Extreme Computational Photography BIBAKFull-Text 1
  Ramesh Raskar
The Camera Culture Group at the MIT Media Lab aims to create a new class of imaging platforms. This talk will discuss three tracks of research: femto photography, retinal imaging, and 3D displays. Femto Photography consists of femtosecond laser illumination, picosecond-accurate detectors and mathematical reconstruction techniques allowing researchers to visualize propagation of light. Direct recording of reflected or scattered light at such a frame rate with sufficient brightness is nearly impossible. Using an indirect 'stroboscopic' method that records millions of repeated measurements by careful scanning in time and viewpoints we can rearrange the data to create a 'movie' of a nanosecond long event. Femto photography and a new generation of nano-photography (using ToF cameras) allow powerful inference with computer vision in presence of scattering. EyeNetra is a mobile phone attachment that allows users to test their own eyesight. The device reveals corrective measures thus bringing vision to billions of people who would not have had access otherwise. Another project, eyeMITRA, is a mobile retinal imaging solution that brings retinal exams to the realm of routine care, by lowering the cost of the imaging device to a 10th of its current cost and integrating the device with image analysis software and predictive analytics. This provides early detection of Diabetic Retinopathy that can change the arc of growth of the world's largest cause of blindness. Finally the talk will describe novel lightfield cameras and lightfield displays that require a compressive optical architecture to deal with high bandwidth requirements of 4D signals.
Keywords: femto photography, retinal imaging, 3D displays, lightfield

Session 1A: Tactile Feedback

GelTouch: Localized Tactile Feedback Through Thin, Programmable Gel BIBAFull-Text 3-10
  Viktor Miruchna; Robert Walter; David Lindlbauer; Maren Lehmann; Regine von Klitzing; Jörg Müller
We present GelTouch, a gel-based layer that can selectively transition between soft and stiff to provide tactile multi-touch feedback. It is flexible, transparent when not activated, and contains no mechanical, electromagnetic, or hydraulic components, resulting in a compact form factor (a 2mm thin touchscreen layer for our prototype). The activated areas can be morphed freely and continuously, without being limited to fixed, predefined shapes. GelTouch consists of a poly(N-isopropylacrylamide) gel layer which alters its viscoelasticity when activated by applying heat (>32°C). We present three different activation techniques: 1) Indium Tin Oxide (ITO) as a heating element that enables tactile feedback through individually addressable taxels; 2) predefined tactile areas of engraved ITO, that can be layered and combined; 3) complex arrangements of resistance wire that create thin tactile edges. We present a tablet with 6x4 tactile areas, enabling a tactile numpad, slider, and thumbstick. We show that the gel is up to 25 times stiffer when activated and that users detect tactile features reliably (94.8%).
Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation BIBAFull-Text 11-19
  Pedro Lopes; Alexandra Ion; Patrick Baudisch
We present impacto, a device designed to render the haptic sensation of hitting or being hit in virtual reality. The key idea that allows the small and light impacto device to simulate a strong hit is that it decomposes the stimulus: it renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impact to the hit by thrusting the user's arm backwards using electrical muscle stimulation. The device is self-contained, wireless, and small enough for wearable use, thus leaves the user unencumbered and able to walk around freely in a virtual environment. The device is of generic shape, allowing it to also be worn on legs, so as to enhance the experience of kicking, or merged into props, such as a baseball bat. We demonstrate how to assemble multiple impacto units into a simple haptic suit. Participants of our study rated impact simulated using impacto's combination of solenoid hit and electrical muscle stimulation as more realistic than either technique in isolation.
Tactile Animation by Direct Manipulation of Grid Displays BIBAFull-Text 21-30
  Oliver S. Schneider; Ali Israr; Karon E. MacLean
Chairs, wearables, and handhelds have become popular sites for spatial tactile display. Visual animators, already expert in using time and space to portray motion, could readily transfer their skills to produce rich haptic sensations if given the right tools. We introduce the tactile animation object, a directly manipulated phantom tactile sensation. This abstraction has two key benefits: 1) efficient, creative, iterative control of spatiotemporal sensations, and 2) the potential to support a variety of tactile grids, including sparse displays. We present Mango, an editing tool for animators, including its rendering pipeline and perceptually-optimized interpolation algorithm for sparse vibrotactile grids. In our evaluation, professional animators found it easy to create a variety of vibrotactile patterns, with both experts and novices preferring the tactile animation object over controlling actuators individually.
Improving Haptic Feedback on Wearable Devices through Accelerometer Measurements BIBAFull-Text 31-36
  Jeffrey R. Blum; Ilja Frissen; Jeremy R. Cooperstock
Many variables have been shown to impact whether a vibration stimulus will be perceived. We present a user study that takes into account not only previously investigated predictors such as vibration intensity and duration along with the age of the person receiving the stimulus, but also the amount of motion, as measured by an accelerometer, at the site of vibration immediately preceding the stimulus. This is a more specific measure than in previous studies showing an effect on perception due to gross conditions such as walking. We show that a logistic regression model including prior acceleration is significantly better at predicting vibration perception than a model including only vibration intensity, duration and participant age. In addition to the overall regression, we discuss individual participant differences and measures of classification performance for real-world applications. Our expectation is that haptic interface designers will be able to use such results to design better vibrations that are perceivable under the user's current activity conditions, without being annoyingly loud or jarring, eventually approaching "perceptually equivalent" feedback independent of motion.

Session 1B: Large Displays, Large Movements

cLuster: Smart Clustering of Free-Hand Sketches on Large Interactive Surfaces BIBAFull-Text 37-46
  Florian Perteneder; Martin Bresler; Eva-Maria Grossauer; Joanne Leong; Michael Haller
Structuring and rearranging free-hand sketches on large interactive surfaces typically requires making multiple stroke selections. This can be both time-consuming and fatiguing in the absence of well-designed selection tools. Investigating the concept of automated clustering, we conducted a background study which highlighted the fact that people have varying perspectives on how elements in sketches can and should be grouped. In response to these diverse user expectations, we present cLuster, a flexible, domain-independent clustering approach for free-hand sketches. Our approach is designed to accept an initial user selection, which is then used to calculate a linear combination of pre-trained perspectives in real-time. The remaining elements are then clustered. An initial evaluation revealed that in many cases, only a few corrections were necessary to achieve the desired clustering results. Finally, we demonstrate the utility of our approach in a variety of application scenarios.
GravitySpot: Guiding Users in Front of Public Displays Using On-Screen Visual Cues BIBAFull-Text 47-56
  Florian Alt; Andreas Bulling; Gino Gravanis; Daniel Buschek
Users tend to position themselves in front of interactive public displays in such a way as to best perceive its content. Currently, this sweet spot is implicitly defined by display properties, content, the input modality, as well as space constraints in front of the display. We present GravitySpot -- an approach that makes sweet spots flexible by actively guiding users to arbitrary target positions in front of displays using visual cues. Such guidance is beneficial, for example, if a particular input technology only works at a specific distance or if users should be guided towards a non-crowded area of a large display. In two controlled lab studies (n=29) we evaluate different visual cues based on color, shape, and motion, as well as position-to-cue mapping functions. We show that both the visual cues and mapping functions allow for fine-grained control over positioning speed and accuracy. Findings are complemented by observations from a 3-month real-world deployment.
Tiltcasting: 3D Interaction on Large Displays using a Mobile Device BIBAFull-Text 57-62
  Krzysztof Pietroszek; James R. Wallace; Edward Lank
We develop and formally evaluate a metaphor for smartphone interaction with 3D environments: Tiltcasting. Under the Tiltcasting metaphor, users interact within a rotatable 2D plane that is "cast" from their phone's interactive display into 3D space. Through an empirical validation, we show that Tiltcasting supports efficient pointing, interaction with occluded objects, disambiguation between nearby objects, and object selection and manipulation in fully addressable 3D space. Our technique out-performs existing target agnostic pointing implementations, and approaches the performance of physical pointing with an off-the-shelf smartphone.
Gunslinger: Subtle Arms-down Mid-air Interaction BIBAFull-Text 63-71
  Mingyu Liu; Mathieu Nancel; Daniel Vogel
We describe Gunslinger, a mid-air interaction technique using barehand postures and gestures. Unlike past work, we explore a relaxed arms-down position with both hands interacting at the sides of the body. It features "hand-cursor" feedback to communicate recognized hand posture, command mode and tracking quality; and a simple, but flexible hand posture recognizer. Although Gunslinger is suitable for many usage contexts, we focus on integrating mid-air gestures with large display touch input. We show how the Gunslinger form factor enables an interaction language that is equivalent, coherent, and compatible with large display touch input. A four-part study evaluates Midas Touch, posture recognition feedback, pointing and clicking, and general usability.

Session 2A: Fabrication 1 -- Augmentation

Encore: 3D Printed Augmentation of Everyday Objects with Printed-Over, Affixed and Interlocked Attachments BIBAFull-Text 73-82
  Xiang 'Anthony' Chen; Stelian Coros; Jennifer Mankoff; Scott E. Hudson
One powerful aspect of 3D printing is its ability to extend, repair, or more generally modify everyday objects. However, nearly all existing work implicitly assumes that whole objects are to be printed from scratch. Designing objects as extensions or enhancements of existing ones is a laborious process in most of today's 3D authoring tools. This paper presents a framework for 3D printing to augment existing objects that covers a wide range of attachment options. We illustrate the framework through three exemplar attachment techniques -- print-over, print-to-affix and print-through, implemented in Encore, a design tool that supports a set of analysis metrics relating to viability, durability and usability that are visualized for the user to explore design options and tradeoffs. Encore also generates 3D models for production, addressing issues such as support jigs and contact geometry between the attached part and the original object. Our validation helps to illustrate the strengths and weaknesses of each technique. For example, print-over is stronger than print-to-affix with adhesives, and all the techniques' strengths are affected by surface curvature.
Patching Physical Objects BIBAFull-Text 83-91
  Alexander Teibrich; Stefanie Mueller; François Guimbretière; Robert Kovacs; Stefan Neubert; Patrick Baudisch
Personal fabrication is currently a one-way process: Once an object has been fabricated with a 3D printer, it cannot be changed anymore; any change requires printing a new version from scratch. The problem is that this approach ignores the nature of design iteration, i.e. that in subsequent iterations large parts of an object stay the same and only small parts change. This makes fabricating from scratch feel unnecessary and wasteful.
   In this paper, we propose a different approach: instead of re-printing the entire object from scratch, we suggest patching the existing object to reflect the next design iteration. We built a system on top of a 3D printer that accomplishes this: Users mount the existing object into the 3D printer, then load both the original and the modified 3D model into our software, which in turn calculates how to patch the object. After identifying which parts to remove and what to add, our system locates the existing object in the printer using the system's built-in 3D scanner. After calibrating the orientation, a mill first removes the outdated geometry, then a print head prints the new geometry in place.
   Since only a fraction of the entire object is refabricated, our approach reduces material consumption and plastic waste (for our example objects by 82% and 93% respectively).
ReForm: Integrating Physical and Digital Design through Bidirectional Fabrication BIBAFull-Text 93-102
  Christian Weichel; John Hardy; Jason Alexander; Hans Gellersen
Digital fabrication machines such as 3D printers and laser-cutters allow users to produce physical objects based on virtual models. The creation process is currently unidirectional: once an object is fabricated it is separated from its originating virtual model. Consequently, users are tied into digital modeling tools, the virtual design must be completed before fabrication, and once fabricated, re-shaping the physical object no longer influences the digital model. To provide a more flexible design process that allows objects to iteratively evolve through both digital and physical input, we introduce bidirectional fabrication. To demonstrate the concept, we built ReForm, a system that integrates digital modeling with shape input, shape output, annotation for machine commands, and visual output. By continually synchronizing the physical object and digital model it supports object versioning to allow physical changes to be undone. Through application examples, we demonstrate the benefits of ReForm to the digital fabrication process.
Makers' Marks: Physical Markup for Designing and Fabricating Functional Objects BIBAFull-Text 103-108
  Valkyrie Savage; Sean Follmer; Jingyi Li; Björn Hartmann
To fabricate functional objects, designers create assemblies combining existing parts (e.g., mechanical hinges, electronic components) with custom-designed geometry (e.g., enclosures). Modeling complex assemblies is outside the reach of the growing number of novice "makers" with access to digital fabrication tools. We aim to allow makers to design and 3D print functional mechanical and electronic assemblies. Based on a formative exploration, we created Makers' Marks, a system based on physically authoring assemblies with sculpting materials and annotation stickers. Makers physically sculpt the shape of an object and attach stickers to place existing parts or high-level features (such as parting lines). Our tool extracts the 3D pose of these annotations from a scan of the design, then synthesizes the geometry needed to support integrating desired parts using a library of clearance and mounting constraints. The resulting designs can then be easily 3D printed and assembled. Our approach enables easy creation of complex objects such as TUIs, and leverages physical materials for tangible manipulation and understanding scale. We validate our tool through several design examples: a custom game controller, an animated toy figure, a friendly baby monitor, and a hinged box with integrated alarm.

Session 2B: 3D & Augmented Reality

Procedural Modeling Using Autoencoder Networks BIBAFull-Text 109-118
  Mehmet Ersin Yumer; Paul Asente; Radomir Mech; Levent Burak Kara
Procedural modeling systems allow users to create high quality content through parametric, conditional or stochastic rule sets. While such approaches create an abstraction layer by freeing the user from direct geometry editing, the nonlinear nature and the high number of parameters associated with such design spaces result in arduous modeling experiences for non-expert users. We propose a method to enable intuitive exploration of such high dimensional procedural modeling spaces within a lower dimensional space learned through autoencoder network training. Our method automatically generates a representative training dataset from the procedural modeling rule set based on shape similarity features. We then leverage the samples in this dataset to train an autoencoder neural network, while also structuring the learned lower dimensional space for continuous exploration with respect to shape features. We demonstrate the efficacy our method with user studies where designers create content with more than 10-fold faster speeds using our system compared to the classic procedural modeling interface.
SHOCam: A 3D Orbiting Algorithm BIBAFull-Text 119-128
  Michael Ortega; Wolfgang Stuerzlinger; Doug Scheurich
In this paper we describe a new orbiting algorithm, called SHOCam, which enables simple, safe and visually attractive control of a camera moving around 3D objects. Compared with existing methods, SHOCam provides a more consistent mapping between the user's interaction and the path of the camera by substantially reducing variability in both camera motion and look direction. Also, we present a new orbiting method that prevents the camera from penetrating object(s), making the visual feedback -- and with it the user experience -- more pleasing and also less error prone. Finally, we present new solutions for orbiting around multiple objects and multi-scale environments.
FoveAR: Combining an Optically See-Through Near-Eye Display with Projector-Based Spatial Augmented Reality BIBAFull-Text 129-135
  Hrvoje Benko; Eyal Ofek; Feng Zheng; Andrew D. Wilson
Optically see-through (OST) augmented reality glasses can overlay spatially-registered computer-generated content onto the real world. However, current optical designs and weight considerations limit their diagonal field of view to less than 40 degrees, making it difficult to create a sense of immersion or give the viewer an overview of the augmented reality space. We combine OST glasses with a projection-based spatial augmented reality display to achieve a novel display hybrid, called FoveAR, capable of greater than 100 degrees field of view, view dependent graphics, extended brightness and color, as well as interesting combinations of public and personal data display. We contribute details of our prototype implementation and an analysis of the interactive design space that our system enables. We also contribute four prototype experiences showcasing the capabilities of FoveAR as well as preliminary user feedback providing insights for enhancing future FoveAR experiences.
Projectibles: Optimizing Surface Color For Projection BIBAFull-Text 137-146
  Brett R. Jones; Rajinder Sodhi; Pulkit Budhiraja; Kevin Karsch; Brian Bailey; David Forsyth
Typically video projectors display images onto white screens, which can result in a washed out image. Projectibles algorithmically control the display surface color to increase the contrast and resolution. By combining a printed image with projected light, we can create animated, high resolution, high dynamic range visual experiences for video sequences. We present two algorithms for separating an input video sequence into a printed component and projected component, maximizing the combined contrast and resolution while minimizing any visual artifacts introduced from the decomposition. We present empirical measurements of real-world results of six example video sequences, subjective viewer feedback ratings, and we discuss the benefits and limitations of Projectibles. This is the first approach to combine a static display with a dynamic display for the display of video, and the first to optimize surface color for projection of video.

Session 3A: Sensing Techniques

Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction BIBAFull-Text 147-156
  Haojian Jin; Christian Holz; Kasper Hornbæk
While current mobile devices detect the presence of surrounding devices, they lack a truly spatial awareness to bring them into the user's natural 3D space. We present Tracko, a 3D tracking system between two or more commodity devices without added components or device synchronization. Tracko achieves this by fusing three signal types. 1) Tracko infers the presence of and rough distance to other devices from the strength of Bluetooth low energy signals. 2) Tracko exchanges a series of inaudible stereo sounds and derives a set of accurate distances between devices from the difference in their arrival times. A Kalman filter integrates both signal cues to place collocated devices in a shared 3D space, combining the robustness of Bluetooth with the accuracy of audio signals for relative 3D tracking. 3) Tracko incorporates inertial sensors to refine 3D estimates and support quick interactions. Tracko robustly tracks devices in 3D with a mean error of 6.5 cm within 0.5 m and a 15.3 cm error within 1 m, which validates Trackoffs suitability for cross-device interactions.
EM-Sense: Touch Recognition of Uninstrumented, Electrical and Electromechanical Objects BIBAFull-Text 157-166
  Gierad Laput; Chouchang Yang; Robert Xiao; Alanson Sample; Chris Harrison
Most everyday electrical and electromechanical objects emit small amounts of electromagnetic (EM) noise during regular operation. When a user makes physical contact with such an object, this EM signal propagates through the user, owing to the conductivity of the human body. By modifying a small, low-cost, software-defined radio, we can detect and classify these signals in real-time, enabling robust on-touch object detection. Unlike prior work, our approach requires no instrumentation of objects or the environment; our sensor is self-contained and can be worn unobtrusively on the body. We call our technique EM-Sense and built a proof-of-concept smartwatch implementation. Our studies show that discrimination between dozens of objects is feasible, independent of wearer, time and local environment.
Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition BIBAFull-Text 167-173
  Yang Zhang; Chris Harrison
We present Tomo, a wearable, low-cost system using Electrical Impedance Tomography (EIT) to recover the interior impedance geometry of a user's arm. This is achieved by measuring the cross-sectional impedances between all pairs of eight electrodes resting on a user's skin. Our approach is sufficiently compact and low-powered that we integrated the technology into a prototype wrist- and armband, which can monitor and classify gestures in real-time. We conducted a user study that evaluated two gesture sets, one focused on gross hand gestures and another using thumb-to-finger pinches. Our wrist location achieved 97% and 87% accuracies on these gesture sets respectively, while our arm location achieved 93% and 81%. We ultimately envision this technique being integrated into future smartwatches, allowing hand gestures and direct touch manipulation to work synergistically to support interactive tasks on small screens.
Corona: Positioning Adjacent Device with Asymmetric Bluetooth Low Energy RSSI Distributions BIBAFull-Text 175-179
  Haojian Jin; Cheng Xu; Kent Lyons
We introduce Corona, a novel spatial sensing technique that implicitly locates adjacent mobile devices in the same plane by examining asymmetric Bluetooth Low Energy RSSI distributions. The underlying phenomenon is that the off-center BLE antenna and asymmetric radio frequency topology create a characteristic Bluetooth RSSI distribution around the device. By comparing the real-time RSSI readings against a RSSI distribution model, each device can derive the relative position of the other adjacent device. Our experiments using an iPhone and iPad Mini show that Corona yields position estimation at 50% accuracy within a 2cm range, or 85% for the best two candidates. We developed an application to combine Corona with accelerometer readings to mitigate ambiguity and enable cross-device interactions on adjacent devices.

Session 3B: Intelligent Information Interfaces

SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries BIBAFull-Text 181-190
  Amy Pavel; Dan B. Goldman; Björn Hartmann; Maneesh Agrawala
Searching for scenes in movies is a time-consuming but crucial task for film studies scholars, film professionals, and new media artists. In pilot interviews we have found that such users search for a wide variety of clips -- e.g., actions, props, dialogue phrases, character performances, locations -- and they return to particular scenes they have seen in the past. Today, these users find relevant clips by watching the entire movie, scrubbing the video timeline, or navigating via DVD chapter menus. Increasingly, users can also index films through transcripts -- however, dialogue often lacks visual context, character names, and high level event descriptions. We introduce SceneSkim, a tool for searching and browsing movies using synchronized captions, scripts and plot summaries. Our interface integrates information from such sources to allow expressive search at several levels of granularity: Captions provide access to accurate dialogue, scripts describe shot-by-shot actions and settings, and plot summaries contain high-level event descriptions. We propose new algorithms for finding word-level caption to script alignments, parsing text scripts, and aligning plot summaries to scripts. Film studies graduate students evaluating SceneSkim expressed enthusiasm about the usability of the proposed system for their research and teaching.
Capture-Time Feedback for Recording Scripted Narration BIBAFull-Text 191-199
  Steve Rubin; Floraine Berthouzoz; Gautham J. Mysore; Maneesh Agrawala
Well-performed audio narrations are a hallmark of captivating podcasts, explainer videos, radio stories, and movie trailers. To record these narrations, professional voiceover actors follow guidelines that describe how to use low-level vocal components -- volume, pitch, timbre, and tempo -- to deliver performances that emphasize important words while maintaining variety, flow and diction. Yet, these techniques are not well-known outside the professional voiceover community, especially among hobbyist producers looking to create their own narrations. We present Narration Coach, an interface that assists novice users in recording scripted narrations. As a user records her narration, our system synchronizes the takes to her script, provides text feedback about how well she is meeting the expert voiceover guidelines, and resynthesizes her recordings to help her hear how she can speak better.
Improving Automated Email Tagging with Implicit Feedback BIBAFull-Text 201-211
  Mohammad S. Sorower; Michael Slater; Thomas G. Dietterich
Tagging email is an important tactic for managing information overload. Machine learning methods can help the user with this task by predicting tags for incoming email messages. The natural user interface displays the predicted tags on the email message, and the user doesn't need to do anything unless those predictions are wrong (in which case, the user can delete the incorrect tags and add the missing tags). From a machine learning perspective, this means that the learning algorithm never receives confirmation that its predictions are correct -- it only receives feedback when it makes a mistake. This can lead to slower learning, particularly when the predictions were not very confident, and hence, the learning algorithm would benefit from positive feedback. One could assume that if the user never changes any tag, then the predictions are correct, but users sometimes forget to correct the tags, presumably because they are focused on the content of the email messages and fail to notice incorrect and missing tags. The aim of this paper is to determine whether implicit feedback can provide useful additional training examples to the email prediction subsystem of TaskTracer, known as EP2 (Email Predictor 2). Our hypothesis is that the more time a user spends working on an email message, the more likely it is that the user will notice tag errors and correct them. If no corrections are made, then perhaps it is safe for the learning system to treat the predicted tags as being correct and train accordingly. This paper proposes three algorithms (and two baselines) for incorporating implicit feedback into the EP2 tag predictor. These algorithms are then evaluated using email interaction and tag correction events collected from 14 user-study participants as they performed email-directed tasks while using TaskTracer EP2. The results show that implicit feedback produces important increases in training feedback, and hence, significant reductions in subsequent prediction errors despite the fact that the implicit feedback is not perfect. We conclude that implicit feedback mechanisms can provide a useful performance boost for email tagging systems.
Codo: Fundraising with Conditional Donations BIBAFull-Text 213-222
  Juan Felipe Beltran; Aysha Siddique; Azza Abouzied; Jay Chen
Crowdfunding websites like Kickstarter and Indiegogo offer project organizers the ability to market, fund, and build a community around their campaign. While offering support and flexibility for organizers, crowdfunding sites provide very little control to donors. In this paper, we investigate the idea of empowering donors by allowing them to specify conditions for their crowdfunding contributions. We introduce a crowdfunding system, Codo, that allows donors to specify conditional donations. Codo allow donors to contribute to a campaign but hold off on their contribution until certain specific conditions are met (e.g. specific members or groups contribute a certain amount). We begin with a micro study to assess several specific conditional donations based on their comprehensibility and usage likelihood. Based on this study, we formalize conditional donations into a general grammar that captures a broad set of useful conditions. We demonstrate the feasibility of resolving conditions in our grammar by elegantly transforming conditional donations into a system of linear inequalities that are efficiently resolved using off-the-shelf linear program solvers. Finally, we designed a user-friendly crowdfunding interface that supports conditional donations for an actual fund raising campaign and assess the potential of conditional donations through this campaign. We find preliminary evidence that roughly 1 in 3 donors make conditional donations and that conditional donors donate more compared to direct donors.

Session 4A: Fabrication 2 -- Flexible and Printed Electronics

Foldio: Digital Fabrication of Interactive and Shape-Changing Objects With Foldable Printed Electronics BIBAFull-Text 223-232
  Simon Olberding; Sergio Soto Ortega; Klaus Hildebrandt; Jürgen Steimle
Foldios are foldable interactive objects with embedded input sensing and output capabilities. Foldios combine the advantages of folding for thin, lightweight and shape-changing objects with the strengths of thin-film printed electronics for embedded sensing and output. To enable designers and end-users to create highly custom interactive foldable objects, we contribute a new design and fabrication approach. It makes it possible to design the foldable object in a standard 3D environment and to easily add interactive high-level controls, eliminating the need to manually design a fold pattern and low-level circuits for printed electronics. Second, we contribute a set of printable user interface controls for touch input and display output on folded objects. Moreover, we contribute controls for sensing and actuation of shape-changeable objects. We demonstrate the versatility of the approach with a variety of interactive objects that have been fabricated with this framework.
uniMorph: Fabricating Thin Film Composites for Shape-Changing Interfaces BIBAFull-Text 233-242
  Felix Heibeck; Basheer Tome; Clark Della Silva; Hiroshi Ishii
Researchers have been investigating shape-changing interfaces, however technologies for thin, reversible shape change remain complicated to fabricate. uniMorph is an enabling technology for rapid digital fabrication of customized thin-film shape-changing interfaces. By combining the thermoelectric characteristics of copper with the high thermal expansion rate of ultra-high molecular weight polyethylene, we are able to actuate the shape of flexible circuit composites directly. The shape-changing actuation is enabled by a temperature driven mechanism and reduces the complexity of fabrication for thin shape-changing interfaces. In this paper we describe how to design and fabricate thin uniMorph composites. We present composites that are actuated by either environmental temperature changes or active heating of embedded structures and provide a systematic overview of shape-changing primitives. Finally, we present different sensing techniques that leverage the existing copper structures or can be seamlessly embedded into the uniMorph composite. To demonstrate the wide applicability of uniMorph, we present several applications in ubiquitous and mobile computing.
Printem: Instant Printed Circuit Boards with Standard Office Printers & Inks BIBAFull-Text 243-251
  Varun Perumal C; Daniel Wigdor
Printem film, a novel method for the fabrication of Printed Circuit Boards (PCBs) for small batch/prototyping use, is presented. Printem film enables a standard office inkjet or laser printer, using standard inks, to produce a PCB: the user prints a negative of the PCB onto the film, exposes it to UV or sunlight, and then tears-away the unneeded portion of the film, leaving-behind a copper PCB. PCBs produced with Printem film are as conductive as PCBs created using standard industrial methods. Herein, the composition of Printem film is described, and advantages of various materials discussed. Sample applications are also described, each of which demonstrates some unique advantage of Printem film over current prototyping methods: conductivity, flexibility, the ability to be cut with a pair of scissors, and the ability to be mounted to a rigid backplane.
   NOTE: publication of full-text held until November 9, 2015.
Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects BIBAFull-Text 253-258
  Martin Schmitz; Mohammadreza Khalilbeigi; Matthias Balwierz; Roman Lissermann; Max Mühlhäuser; Jürgen Steimle
3D printing is widely used to physically prototype the look and feel of 3D objects. Interaction possibilities of these prototypes, however, are often limited to mechanical parts or post-assembled electronics. In this paper, we present Capricate, a fabrication pipeline that enables users to easily design and 3D print highly customized objects that feature embedded capacitive multi-touch sensing. The object is printed in a single pass using a commodity multi-material 3D printer. To enable touch input on a wide variety of 3D printable surfaces, we contribute two techniques for designing and printing embedded sensors of custom shape. The fabrication pipeline is technically validated by a series of experiments and practically validated by a set of example applications. They demonstrate the wide applicability of Capricate for interactive objects.

Session 4B: Tools for Programmers

Explaining Visual Changes in Web Interfaces BIBAFull-Text 259-268
  Brian Burg; Andrew J. Ko; Michael D. Ernst
Web developers often want to repurpose interactive behaviors from third-party web pages, but struggle to locate the specific source code that implements the behavior. This task is challenging because developers must find and connect all of the non-local interactions between event-based JavaScript code, declarative CSS styles, and web page content that combine to express the behavior.
   The Scry tool embodies a new approach to locating the code that implements interactive behaviors. A developer selects a page element; whenever the element changes, Scry captures the rendering engine's inputs (DOM, CSS) and outputs (screenshot) for the element. For any two captured element states, Scry can compute how the states differ and which lines of JavaScript code were responsible. Using Scry, a developer can locate an interactive behavior's implementation by picking two output states; Scry indicates the JavaScript code directly responsible for their differences.
Unravel: Rapid Web Application Reverse Engineering via Interaction Recording, Source Tracing, and Library Detection BIBAFull-Text 270-279
  Joshua Hibschman; Haoqi Zhang
Professional websites with complex UI features provide real world examples for developers to learn from. Yet despite the availability of source code, it is still difficult to understand how these features are implemented. Existing tools such as the Chrome Developer Tools and Firebug offer debugging and inspection, but reverse engineering is still a time consuming task. We thus present Unravel, an extension of the Chrome Developer Tools for quickly tracking and visualizing HTML changes, JavaScript method calls, and JavaScript libraries. Unravel injects an observation agent into websites to monitor DOM interactions in real-time without functional interference or external dependencies. To manage potentially large observations of events, the Unravel UI provides affordances to reduce, sort, and scope observations. Testing Unravel with 13 web developers on 5 large-scale websites, we found a 53% decrease in time to discovering the first key source behind a UI feature and a 32% decrease in time to understanding how to fully recreate a feature.
Webstrates: Shareable Dynamic Media BIBAFull-Text 280-290
  Clemens N. Klokmose; James R. Eagan; Siemen Baader; Wendy Mackay; Michel Beaudouin-Lafon
We revisit Alan Kay's early vision of dynamic media that blurs the distinction between documents and applications. We introduce shareable dynamic media that are malleable by users, who may appropriate them in idiosyncratic ways; shareable among users, who collaborate on multiple aspects of the media; and distributable across diverse devices and platforms. We present Webstrates, an environment for exploring shareable dynamic media. Webstrates augment web technology with real-time sharing. They turn web pages into substrates, i.e. software entities that act as applications or documents depending upon use. We illustrate Webstrates with two implemented case studies: users collaboratively author an article with functionally and visually different editors that they can personalize and extend at run-time; and they orchestrate its presentation and audience participation with multiple devices. We demonstrate the simplicity and generative power of Webstrates with three additional prototypes and evaluate it from a systems perspective.
User Interaction Models for Disambiguation in Programming by Example BIBAFull-Text 291-301
  Mikaël Mayer; Gustavo Soares; Maxim Grechkin; Vu Le; Mark Marron; Oleksandr Polozov; Rishabh Singh; Benjamin Zorn; Sumit Gulwani
Programming by Examples (PBE) has the potential to revolutionize end-user programming by enabling end users, most of whom are non-programmers, to create small scripts for automating repetitive tasks. However, examples, though often easy to provide, are an ambiguous specification of the user's intent. Because of that, a key impedance in adoption of PBE systems is the lack of user confidence in the correctness of the program that was synthesized by the system. We present two novel user interaction models that communicate actionable information to the user to help resolve ambiguity in the examples. One of these models allows the user to effectively navigate between the huge set of programs that are consistent with the examples provided by the user. The other model uses active learning to ask directed example-based questions to the user on the test input data over which the user intends to run the synthesized program. Our user studies show that each of these models significantly reduces the number of errors in the performed task without any difference in completion time. Moreover, both models are perceived as useful, and the proactive active-learning based model has a slightly higher preference regarding the users' confidence in the result.

Session 5A: Touch Input

Biometric Touch Sensing: Seamlessly Augmenting Each Touch with Continuous Authentication BIBAFull-Text 303-312
  Christian Holz; Marius Knaust
Current touch devices separate user authentication from regular interaction, for example by displaying modal login screens before device usage or prompting for in-app passwords, which interrupts the interaction flow. We propose biometric touch sensing, a new approach to representing touch events that enables commodity devices to seamlessly integrate authentication into interaction: From each touch, the touchscreen senses the 2D input coordinates and at the same time obtains biometric features that identify the user. Our approach makes authentication during interaction transparent to the user, yet ensures secure interaction at all times. To implement this on today's devices, our watch prototype Bioamp senses the impedance profile of the user's wrist and modulates a signal onto the user's body through skin using a periodic electric signal. This signal affects the capacitive values touchscreens measure upon touch, allowing devices to identify users on each touch. We integrate our approach into Windows 8 and discuss and demonstrate it in the context of various use cases, including access permissions and protecting private screen contents on personal and shared devices.
Push-Push: A Drag-like Operation Overlapped with a Page Transition Operation on Touch Interfaces BIBAFull-Text 313-322
  Jaehyun Han; Geehyuk Lee
A page transition operation on touch interfaces is a common and frequent subtask when one conducts a drag-like operation such as selecting text and dragging an icon. Traditional page transition gestures such as scrolling and flicking gestures, however, cannot be conducted while conducting the drag-like operation since they have a confliction. We proposed Push-Push that is a new drag-like operation not in conflict with page transition operations. Thus, page transition operations could be conducted while performing Push-Push. To design Push-Push, we utilized the hover and pressed states as additional input states of touch interfaces. The results from two experiments showed that Push-Push has an advantage on increasing performance and qualitative opinions of users while reducing the subjective overload.
Pin-and-Cross: A Unimanual Multitouch Technique Combining Static Touches with Crossing Selection BIBAFull-Text 323-332
  Yuexing Luo; Daniel Vogel
We define, explore, and demonstrate a new multitouch interaction space called "pin-and-cross." It combines one or more static touches ("pins") with another touch to cross a radial target, all performed with one hand. A formative study reveals pin-and-cross kinematic characteristics and evaluates fundamental performance and preference for target angles. These results are used to form design guidelines and recognition heuristics for pin-and-cross menus invoked with one and two pin fingers on first touch or after a drag. These guidelines are used to implement different pin-and-cross techniques. A controlled experiment compares a one finger pin-and-cross contextual menu to a Marking Menu and partial Pie Menu: pin-and-cross is just as accurate and 27% faster when invoked on a draggable object. A photo app demonstrates more pin-and-cross variations for extending two-finger scrolling, selecting modes while drawing, constraining two-finger transformations, and combining pin-and-cross with a Marking Menu.

Session 5B: Tangibles

LineFORM: Actuated Curve Interfaces for Display, Interaction, and Constraint BIBAFull-Text 333-339
  Ken Nakagaki; Sean Follmer; Hiroshi Ishii
In this paper we explore the design space of actuated curve interfaces, a novel class of shape changing-interfaces. Physical curves have several interesting characteristics from the perspective of interaction design: they have a variety of inherent affordances; they can easily represent abstract data; and they can act as constraints, boundaries, or borderlines. By utilizing such aspects of lines and curves, together with the added capability of shape-change, new possibilities for display, interaction and body constraint are possible. In order to investigate these possibilities we have implemented two actuated curve interfaces at different scales. LineFORM, our implementation, inspired by serpentine robotics, is comprised of a series chain of 1DOF servo motors with integrated sensors for direct manipulation. To motivate this work we present various applications such as shape changing cords, mobiles, body constraints, and data manipulation tools.
Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display BIBAFull-Text 341-349
  Philipp Schoessler; Daniel Windham; Daniel Leithinger; Sean Follmer; Hiroshi Ishii
Pin-based shape displays not only give physical form to digital information, they have the inherent ability to accurately move and manipulate objects placed on top of them. In this paper we focus on such object manipulation: we present ideas and techniques that use the underlying shape change to give kinetic ability to otherwise inanimate objects. First, we describe the shape display's ability to assemble, disassemble, and reassemble structures from simple passive building blocks through stacking, scaffolding, and catapulting. A technical evaluation demonstrates the reliability of the presented techniques. Second, we introduce special kinematic blocks that are actuated and sensed through the underlying pins. These blocks translate vertical pin movements into other degrees of freedom like rotation or horizontal movement. This interplay of the shape display with objects on its surface allows us to render otherwise inaccessible forms, like overhangs, and enables richer input and output.
PERCs: Persistently Trackable Tangibles on Capacitive Multi-Touch Displays BIBAFull-Text 351-356
  Simon Voelker; Christian Cherek; Jan Thar; Thorsten Karrer; Christian Thoresen; Kjell Ivar Øvergård; Jan Borchers
Tangible objects on capacitive multi-touch surfaces are usually only detected while the user is touching them. When the user lets go of such a tangible, the system cannot distinguish whether the user just released the tangible, or picked it up and removed it from the surface. We introduce PERCs, persistent capacitive tangibles that "know" whether they are currently on a capacitive touch surface or not. This is achieved by adding a small field sensor to the tangible to detect the touch screen's own, weak electromagnetic touch detection probing signal. Thus, unlike previous designs, PERCs do not get filtered out over time by the adaptive signal filters of the touch screen. We provide a technical overview of the theory behind PERCs and our prototype construction, and we evaluate detection rates, timing performance, and positional and angular accuracy for PERCs on a variety of unmodified, commercially available multi-touch devices. Through their affordable circuitry and high accuracy, PERCs open up the potential for a variety of new applications that use tangibles on today's ubiquitous multi-touch devices.
SmartTokens: Embedding Motion and Grip Sensing in Small Tangible Objects BIBAFull-Text 357-362
  Mathieu Le Goc; Pierre Dragicevic; Samuel Huron; Jeremy Boy; Jean-Daniel Fekete
SmartTokens are small-sized tangible tokens that can sense multiple types of motion, multiple types of touch/grip, and send input events wirelessly as state-machine transitions. By providing an open platform for embedding basic sensing capabilities within small form-factors, SmartTokens extend the design space of tangible user interfaces. We describe the design and implementation of SmartTokens and illustrate how they can be used in practice by introducing a novel TUI design for event notification and personal task management.

Session 6A: Gaze

Self-Calibrating Head-Mounted Eye Trackers Using Egocentric Visual Saliency BIBAFull-Text 363-372
  Yusuke Sugano; Andreas Bulling
Head-mounted eye tracking has significant potential for gaze-based applications such as life logging, mental health monitoring, or the quantified self. A neglected challenge for the long-term recordings required by these applications is that drift in the initial person-specific eye tracker calibration, for example caused by physical activity, can severely impact gaze estimation accuracy and thus system performance and user experience. We first analyse calibration drift on a new dataset of natural gaze data recorded using synchronised video-based and Electrooculography-based eye trackers of 20 users performing everyday activities in a mobile setting. Based on this analysis we present a method to automatically self-calibrate head-mounted eye trackers based on a computational model of bottom-up visual saliency. Through evaluations on the dataset we show that our method 1) is effective in reducing calibration drift in calibrated eye trackers and 2) given sufficient data, can achieve gaze estimation accuracy competitive with that of a calibrated eye tracker, without any manual calibration.
Gaze-Shifting: Direct-Indirect Input with Pen and Touch Modulated by Gaze BIBAFull-Text 373-383
  Ken Pfeuffer; Jason Alexander; Ming Ki Chong; Yanxia Zhang; Hans Gellersen
Modalities such as pen and touch are associated with direct input but can also be used for indirect input. We propose to combine the two modes for direct-indirect input modulated by gaze. We introduce gaze-shifting as a novel mechanism for switching the input mode based on the alignment of manual input and the user's visual attention. Input in the user's area of attention results in direct manipulation whereas input offset from the user's gaze is redirected to the visual target. The technique is generic and can be used in the same manner with different input modalities. We show how gaze-shifting enables novel direct-indirect techniques with pen, touch, and combinations of pen and touch input.
Gaze vs. Mouse: A Fast and Accurate Gaze-Only Click Alternative BIBAFull-Text 385-394
  Christof Lutteroth; Moiz Penkar; Gerald Weber
Eye gaze tracking is a promising input method which is gradually finding its way into the mainstream. An obvious question to arise is whether it can be used for point-and-click tasks, as an alternative for mouse or touch. Pointing with gaze is both fast and natural, although its accuracy is limited. There are still technical challenges with gaze tracking, as well as inherent physiological limitations. Furthermore, providing an alternative to clicking is challenging.
   We are considering use cases where input based purely on gaze is desired, and the click targets are discrete user interface (UI) elements which are too small to be reliably resolved by gaze alone, e.g., links in hypertext. We present Actigaze, a new gaze-only click alternative which is fast and accurate for this scenario. A clickable user interface element is selected by dwelling on one of a set of confirm buttons, based on two main design contributions: First, the confirm buttons stay on fixed positions with easily distinguishable visual identifiers such as colors, enabling procedural learning of the confirm button position. Secondly, UI elements are associated with confirm buttons through the visual identifiers in a way which minimizes the likelihood of inadvertent clicks. We evaluate two variants of the proposed click alternative, comparing them against the mouse and another gaze-only click alternative.
GazeProjector: Accurate Gaze Estimation and Seamless Gaze Interaction Across Multiple Displays BIBAFull-Text 395-404
  Christian Lander; Sven Gehring; Antonio Krüger; Sebastian Boring; Andreas Bulling
Mobile gaze-based interaction with multiple displays may occur from arbitrary positions and orientations. However, maintaining high gaze estimation accuracy in such situations remains a significant challenge. In this paper, we present GazeProjector, a system that combines (1) natural feature tracking on displays to determine the mobile eye tracker's position relative to a display with (2) accurate point-of-gaze estimation. GazeProjector allows for seamless gaze estimation and interaction on multiple displays of arbitrary sizes independently of the user's position and orientation to the display. In a user study with 12 participants we compare GazeProjector to established methods (here: visual on-screen markers and a state-of-the-art video-based motion capture system). We show that our approach is robust to varying head poses, orientations, and distances to the display, while still providing high gaze estimation accuracy across multiple displays without recalibration for each variation. Our system represents an important step towards the vision of pervasive gaze-based interfaces.

Session 6B: Pushing Virtual and Physical Envelopes

Virtual Replicas for Remote Assistance in Virtual and Augmented Reality BIBAFull-Text 405-415
  Ohan Oda; Carmine Elvezio; Mengu Sukan; Steven Feiner; Barbara Tversky
In many complex tasks, a remote subject-matter expert may need to assist a local user to guide actions on objects in the local user's environment. However, effective spatial referencing and action demonstration in a remote physical environment can be challenging. We introduce two approaches that use Virtual Reality (VR) or Augmented Reality (AR) for the remote expert, and AR for the local user, each wearing a stereo head-worn display. Both approaches allow the expert to create and manipulate virtual replicas of physical objects in the local environment to refer to parts of those physical objects and to indicate actions on them. This can be especially useful for parts that are occluded or difficult to access. In one approach, the expert points in 3D to portions of virtual replicas to annotate them. In another approach, the expert demonstrates actions in 3D by manipulating virtual replicas, supported by constraints and annotations. We performed a user study of a 6DOF alignment task, a key operation in many physical task domains, comparing both approaches to an approach in which the expert uses a 2D tablet-based drawing system similar to ones developed for prior work on remote assistance. The study showed the 3D demonstration approach to be faster than the others. In addition, the 3D pointing approach was faster than the 2D tablet in the case of a highly trained expert.
TurkDeck: Physical Virtual Reality Based on People BIBAFull-Text 417-426
  Lung-Pan Cheng; Thijs Roumen; Hannes Rantzsch; Sven Köhler; Patrick Schmidt; Robert Kovacs; Johannes Jasper; Jonas Kemper; Patrick Baudisch
TurkDeck is an immersive virtual reality system that reproduces not only what users see and hear, but also what users feel. TurkDeck produces the haptic sensation using props, i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. Unlike previous work on prop-based virtual reality, however, TurkDeck allows creating arbitrarily large virtual worlds in finite space and using a finite set of physical props. The key idea behind TurkDeck is that it creates these physical representations on the fly by making a group of human workers present and operate the props only when and where the user can actually reach them. TurkDeck manages these so-called "human actuators" by displaying visual instructions that tell the human actuators when and where to place props and how to actuate them. We demonstrate TurkDeck at the example of an immersive 300m2 experience in 25m2 physical space. We show how to simulate a wide range of physical objects and effects, including walls, doors, ledges, steps, beams, switches, stompers, portals, zip lines, and wind. In a user study, participants rated the realism/immersion of TurkDeck higher than a traditional prop-less baseline condition (4.9 vs. 3.6 on 7 item Likert).
Protopiper: Physically Sketching Room-Sized Objects at Actual Scale BIBAFull-Text 427-436
  Harshit Agrawal; Udayan Umapathi; Robert Kovacs; Johannes Frohnhofen; Hsiang-Ting Chen; Stefanie Mueller; Patrick Baudisch
Physical sketching of 3D wireframe models, using a hand-held plastic extruder, allows users to explore the design space of 3D models efficiently. Unfortunately, the scale of these devices limits users' design explorations to small-scale objects. We present protopiper, a computer aided, hand-held fabrication device, that allows users to sketch room-sized objects at actual scale. The key idea behind protopiper is that it forms adhesive tape into tubes as its main building material, rather than extruded plastic or photopolymer lines. Since the resulting tubes are hollow they offer excellent strength-to-weight ratio, thus scale well to large structures. Since the tape is pre-coated with adhesive it allows connecting tubes quickly, unlike extruded plastic that would require heating and cooling in the kilowatt range. We demonstrate protopiper's use through several demo objects, ranging from more constructive objects, such as furniture, to more decorative objects, such as statues. In our exploratory user study, 16 participants created objects based on their own ideas. They rated the device as being "useful for creative exploration", "its ability to sketch at actual scale helped judge fit", and "fun to use."
RevoMaker: Enabling Multi-directional and Functionally-embedded 3D printing using a Rotational Cuboidal Platform BIBAFull-Text 437-446
  Wei Gao; Yunbo Zhang; Diogo C. Nazzetta; Karthik Ramani; Raymond J. Cipra
In recent years, 3D printing has gained significant attention from the maker community, academia, and industry to support low-cost and iterative prototyping of designs. Current unidirectional extrusion systems require printing sacrificial material to support printed features such as overhangs. Furthermore, integrating functions such as sensing and actuation into these parts requires additional steps and processes to create "functional enclosures", since design functionality cannot be easily embedded into prototype printing. All of these factors result in relatively high design iteration times. We present "RevoMaker", a self-contained 3D printer that creates direct out-of-the-printer functional prototypes, using less build material and with substantially less reliance on support structures. By modifying a standard low-cost FDM printer with a revolving cuboidal platform and printing partitioned geometries around cuboidal facets, we achieve a multidirectional additive prototyping process to reduce the print and support material use. Our optimization framework considers various orientations and sizes for the cuboidal base. The mechanical, electronic, and sensory components are preassembled on the flattened laser-cut facets and enclosed inside the cuboid when closed. We demonstrate RevoMaker directly printing a variety of customized and fully-functional product prototypes, such as computer mice and toys, thus illustrating the new affordances of 3D printing for functional product design.

Session 7A: Wearable and Mobile Interactions

NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus BIBAFull-Text 447-456
  Haijun Xia; Tovi Grossman; George Fitzmaurice
Due to their limited input area, ultra-small devices, such as smartwatches, are even more prone to occlusion or the fat finger problem, than their larger counterparts, such as smart phones, tablets, and tabletop displays. We present NanoStylus -- a finger-mounted fine-tip stylus that enables fast and accurate pointing on a smartwatch with almost no occlusion. The NanoStylus is built from the circuitry of an active capacitive stylus, and mounted within a custom 3D-printed thimble-shaped housing unit. A sensor strip is mounted on each side of the device to enable additional gestures. A user study shows that NanoStylus reduces error rate by 80%, compared to traditional touch interaction and by 45%, compared to a traditional stylus. This high precision pointing capability, coupled with the implemented gesture sensing, gives us the opportunity to explore a rich set of interactive applications on a smartwatch form factor.
Orbits: Gaze Interaction for Smart Watches using Smooth Pursuit Eye Movements BIBAFull-Text 457-466
  Augusto Esteves; Eduardo Velloso; Andreas Bulling; Hans Gellersen
We introduce Orbits, a novel gaze interaction technique that enables hands-free input on smart watches. The technique relies on moving controls to leverage the smooth pursuit movements of the eyes and detect whether and at which control the user is looking at. In Orbits, controls include targets that move in a circular trajectory in the face of the watch, and can be selected by following the desired one for a small amount of time. We conducted two user studies to assess the technique's recognition and robustness, which demonstrated how Orbits is robust against false positives triggered by natural eye movements and how it presents a hands-free, high accuracy way of interacting with smart watches using off-the-shelf devices. Finally, we developed three example interfaces built with Orbits: a music player, a notifications face plate and a missed call menu. Despite relying on moving controls -- very unusual in current HCI interfaces -- these were generally well received by participants in a third and final study.
Candid Interaction: Revealing Hidden Mobile and Wearable Computing Activities BIBAFull-Text 467-476
  Barrett Ens; Tovi Grossman; Fraser Anderson; Justin Matejka; George Fitzmaurice
The growth of mobile and wearable technologies has made it often difficult to understand what people in our surroundings are doing with their technology. In this paper, we introduce the concept of candid interaction: techniques for providing awareness about our mobile and wearable device usage to others in the vicinity. We motivate and ground this exploration through a survey on current attitudes toward device usage during interpersonal encounters. We then explore a design space for candid interaction through seven prototypes that leverage a wide range of technological enhancements, such as Augmented Reality, shape memory muscle wire, and wearable projection. Preliminary user feedback of our prototypes highlights the trade-offs between the benefits of sharing device activity and the need to protect user privacy.
Sensing Tablet Grasp + Micro-mobility for Active Reading BIBAFull-Text 477-487
  Dongwook Yoon; Ken Hinckley; Hrvoje Benko; François Guimbretière; Pourang Irani; Michel Pahud; Marcel Gavriliu
The orientation and repositioning of physical artefacts (such as paper documents) to afford shared viewing of content, or to steer the attention of others to specific details, is known as micro-mobility. But the role of grasp in micro-mobility has rarely been considered, much less sensed by devices. We therefore employ capacitive grip sensing and inertial motion to explore the design space of combined grasp + micro-mobility by considering three classes of technique in the context of active reading. Single user, single device techniques support grip-influenced behaviors such as bookmarking a page with a finger, but combine this with physical embodiment to allow flipping back to a previous location. Multiple user, single device techniques, such as passing a tablet to another user or working side-by-side on a single device, add fresh nuances of expression to co-located collaboration. And single user, multiple device techniques afford facile cross-referencing of content across devices. Founded on observations of grasp and micro-mobility, these techniques open up new possibilities for both individual and collaborative interaction with electronic documents.

Session 7B: Neurons, Affect, Ambiguity

DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization BIBAFull-Text 489-500
  Tong Gao; Mira Dontcheva; Eytan Adar; Zhicheng Liu; Karrie G. Karahalios
Answering questions with data is a difficult and time-consuming process. Visual dashboards and templates make it easy to get started, but asking more sophisticated questions often requires learning a tool designed for expert analysts. Natural language interaction allows users to ask questions directly in complex programs without having to learn how to use an interface. However, natural language is often ambiguous. In this work we propose a mixed-initiative approach to managing ambiguity in natural language interfaces for data visualization. We model ambiguity throughout the process of turning a natural language query into a visualization and use algorithmic disambiguation coupled with interactive ambiguity widgets. These widgets allow the user to resolve ambiguities by surfacing system decisions at the point where the ambiguity matters. Corrections are stored as constraints and influence subsequent queries. We have implemented these ideas in a system, DataTone. In a comparative study, we find that DataTone is easy to learn and lets users ask questions without worrying about syntax and proper question form.
On Sounder Ground: CAAT, a Viable Widget for Affective Reaction Assessment BIBAFull-Text 501-510
  Bruno Cardoso; Osvaldo Santos; Teresa Romão
The reliable assessment of affective reactions to stimuli is paramount in a variety of scientific fields, including HCI (Human-Computer Interaction). Variation of emotional states over time, however, warrants the need for quick measurements of emotions. To address it, new tools for quick assessments of affective states have been developed. In this work, we explore the CAAT (Circumplex Affective Assessment Tool), an instrument with a unique design in the scope of affect assessment -- a graphical control element -- that makes it amenable to seamless integration in user interfaces. We briefly describe the CAAT and present a multi-dimensional evaluation that evidences the tool's viability. We have assessed its test-retest reliability, construct validity and quickness of use, by collecting data through an unsupervised, web-based user study. Results show high test-retest reliability, evidence the tool's construct validity and confirm its quickness of use, making it a good fit for longitudinal studies and systems requiring quick assessments of emotional reactions.
Anger-based BCI Using fNIRS Neurofeedback BIBAFull-Text 511-521
  Gabor Aranyi; Fred Charles; Marc Cavazza
Functional near-infrared spectroscopy (fNIRS) holds increasing potential for Brain-Computer Interfaces (BCI) due to its portability, ease of application, robustness to movement artifacts, and relatively low cost. The use of fNIRS to support the development of affective BCI has received comparatively less attention, despite the role played by the prefrontal cortex in affective control, and the appropriateness of fNIRS to measure prefrontal activity. We present an active, fNIRS-based neurofeedback (NF) interface, which uses differential changes in oxygenation between the left and right sides of the dorsolateral prefrontal cortex to operationalize BCI input. The system is activated by users generating a state of anger, which has been previously linked to increased left prefrontal asymmetry. We have incorporated this NF interface into an experimental platform adapted from a virtual 3D narrative, in which users can express anger at a virtual character perceived as evil, causing the character to disappear progressively. Eleven subjects used the system and were able to successfully perform NF despite minimal training. Extensive analysis confirms that success was associated with the intent to express anger. This has positive implications for the design of affective BCI based on prefrontal asymmetry.
Leveraging Dual-Observable Input for Fine-Grained Thumb Interaction Using Forearm EMG BIBAFull-Text 523-528
  Donny Huang; Xiaoyi Zhang; T. Scott Saponas; James Fogarty; Shyamnath Gollakota
We introduce the first forearm-based EMG input system that can recognize fine-grained thumb gestures, including left swipes, right swipes, taps, long presses, and more complex thumb motions. EMG signals for thumb motions sensed from the forearm are quite weak and require significant training data to classify. We therefore also introduce a novel approach for minimally-intrusive collection of labeled training data for always-available input devices. Our dual-observable input approach is based on the insight that interaction observed by multiple devices allows recognition by a primary device (e.g., phone recognition of a left swipe gesture) to create labeled training examples for another (e.g., forearm-based EMG data labeled as a left swipe). We implement a wearable prototype with dry EMG electrodes, train with labeled demonstrations from participants using their own phones, and show that our prototype can recognize common fine-grained thumb gestures and user-defined complex gestures.

Session 8A: Hands and Fingers

Improving Virtual Keyboards When All Finger Positions Are Known BIBAFull-Text 529-538
  Daewoong Choi; Hyeonjoong Cho; Joono Cheong
Current virtual keyboards are known to be slower and less convenient than physical QWERTY keyboards because they simply imitate the traditional QWERTY keyboards on touchscreens. In order to improve virtual keyboards, we consider two reasonable assumptions based on the observation of skilled typists. First, the keys are already assigned to each finger for typing. Based on this assumption, we suggest restricting each finger to entering pre-allocated keys only. Second, non-touching fingers move in correlation with the touching finger because of the intrinsic structure of human hands. To verify of our assumptions, we conducted two experiments with skilled typists. In the first experiment, we statistically verified the second assumption. We then suggest a novel virtual keyboard using our observations. In the second experiment, we show that our suggested keyboard outperforms existing virtual keyboards.
ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data BIBAFull-Text 539-548
  Xin Yi; Chun Yu; Mingrui Zhang; Sida Gao; Ke Sun; Yuanchun Shi
Ten-finger freehand mid-air typing is a potential solution for post-desktop interaction. However, the absence of tactile feedback as well as the inability to accurately distinguish tapping finger or target keys exists as the major challenge for mid-air typing. In this paper, we present ATK, a novel interaction technique that enables freehand ten-finger typing in the air based on 3D hand tracking data. Our hypothesis is that expert typists are able to transfer their typing ability from physical keyboards to mid-air typing. We followed an iterative approach in designing ATK. We first empirically investigated users' mid-air typing behavior, and examined fingertip kinematics during tapping, correlated movement among fingers and 3D distribution of tapping endpoints. Based on the findings, we proposed a probabilistic tap detection algorithm, and augmented Goodman's input correction model to account for the ambiguity in distinguishing tapping finger. We finally evaluated the performance of ATK with a 4-block study. Participants typed 23.0 WPM with an uncorrected word-level error rate of 0.3% in the first block, and later achieved 29.2 WPM in the last block without sacrificing accuracy.
CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring BIBAFull-Text 549-556
  Liwei Chan; Yi-Ling Chen; Chi-Hao Hsieh; Rong-Hao Liang; Bing-Yu Chen
This paper presents CyclopsRing, a ring-style fisheye imaging wearable device that can be worn on hand webbings to enable whole-hand and context-aware interactions. Observing from a central position of the hand through a fisheye perspective, CyclopsRing sees not only the operating hand, but also the environmental contexts that involve with the hand-based interactions. Since CyclopsRing is a finger-worn device, it also allows users to fully preserve skin feedback of the hands. This paper demonstrates a proof-of-concept device, reports the performance in hand-gesture recognition using random decision forest (RDF) method, and, upon the gesture recognizer, presents a set of interaction techniques including on-finger pinch-and-slide input, in-air pinch-and-motion input, palm-writing input, and their interactions with the environmental contexts. The experiment obtained an 84.75% recognition rate of hand gesture input from a database of seven hand gestures collected from 15 participants. To our knowledge, CyclopsRing is the first ring-wearable device that supports whole-hand and context-aware interactions.
BackHand: Sensing Hand Gestures via Back of the Hand BIBAFull-Text 557-564
  Jhe-Wei Lin; Chiuan Wang; Yi Yao Huang; Kuan-Ting Chou; Hsuan-Yu Chen; Wei-Luan Tseng; Mike Y. Chen
In this paper, we explore using the back of hands for sensing hand gestures, which interferes less than glove-based approaches and provides better recognition than sensing at wrists and forearms. Our prototype, BackHand, uses an array of strain gauge sensors affixed to the back of hands, and applies machine learning techniques to recognize a variety of hand gestures. We conducted a user study with 10 participants to better understand gesture recognition accuracy and the effects of sensing locations. Results showed that sensor reading patterns differ significantly across users, but are consistent for the same user. The leave-one-user-out accuracy is low at an average of 27.4%, but reaches 95.8% average accuracy for 16 popular hand gestures when personalized for each participant. The most promising location spans the 1/8~1/4 area between the metacarpophalangeal joints (MCP, the knuckles between the hand and fingers) and the head of ulna (tip of the wrist).

Session 8B: Fabrication 3 -- Complex Shapes and Properties

MoveableMaker: Facilitating the Design, Generation, and Assembly of Moveable Papercraft BIBAFull-Text 565-574
  Michelle Annett; Tovi Grossman; Daniel Wigdor; George Fitzmaurice
In this work, we explore moveables, i.e., interactive papercraft that harness user interaction to generate visual effects. First, we present a survey of children's books that captured the state of the art of moveables. The results of this survey were synthesized into a moveable taxonomy and informed MoveableMaker, a new tool to assist users in designing, generating, and assembling moveable papercraft. MoveableMaker supports the creation and customization of a number of moveable effects and employs moveable-specific features including animated tooltips, automatic instruction generation, constraint-based rendering, techniques to reduce material waste, and so on. To understand how MoveableMaker encourages creativity and enhances the workflow when creating moveables, a series of exploratory workshops were conducted. The results of these explorations, including the content participants created and their impressions, are discussed, along with avenues for future research involving moveables.
LaserStacker: Fabricating 3D Objects by Laser Cutting and Welding BIBAFull-Text 575-582
  Udayan Umapathi; Hsiang-Ting Chen; Stefanie Mueller; Ludwig Wall; Anna Seufert; Patrick Baudisch
Laser cutters are useful for rapid prototyping because they are fast. However, they only produce planar 2D geometry. One approach to creating non-planar objects is to cut the object in horizontal slices and to stack and glue them. This approach, however, requires manual effort for the assembly and time for the glue to set, defeating the purpose of using a fast fabrication tool. We propose eliminating the assembly step with our system LaserStacker. The key idea is to use the laser cutter to not only cut but also to weld. Users place not one acrylic sheet, but a stack of acrylic sheets into their cutter. In a single process, LaserStacker cuts each individual layer to shape (through all layers above it), welds layers by melting material at their interface, and heals undesired cuts in higher layers. When users take out the object from the laser cutter, it is already assembled. To allow users to model stacked objects efficiently, we built an extension to a commercial 3D editor (SketchUp) that provides tools for defining which parts should be connected and which remain loose. When users hit the export button, LaserStacker converts the 3D model into cutting, welding, and healing instructions for the laser cutter. We show how LaserStacker does not only allow making static objects, such as architectural models, but also objects with moving parts and simple mechanisms, such as scissors, a simple pinball machine, and a mechanical toy with gears.
HapticPrint: Designing Feel Aesthetics for Digital Fabrication BIBAFull-Text 583-591
  Cesar Torres; Tim Campbell; Neil Kumar; Eric Paulos
Digital fabrication has enabled massive creativity in hobbyist communities and professional product design. These emerging technologies excel at realizing an arbitrary shape or form; however these objects are often rigid and lack the feel desired by designers. We aim to enable physical haptic design in passive 3D printed objects. This paper identifies two core areas for extending physical design into digital fabrication: designing the external and internal haptic characteristics of an object. We present HapticPrint as a pair of design tools to easily modify the feel of a 3D model. Our external tool maps textures and UI elements onto arbitrary shapes, and our internal tool modifies the internal geometry of models for novel compliance and weight characteristics. We demonstrate the value of HapticPrint with a range of applications that expand the aesthetics of feel, usability, and interactivity in 3D artifacts.
3D Printed Hair: Fused Deposition Modeling of Soft Strands, Fibers, and Bristles BIBAFull-Text 593-597
  Gierad Laput; Xiang 'Anthony' Chen; Chris Harrison
We introduce a technique for furbricating 3D printed hair, fibers and bristles, by exploiting the stringing phenomena inherent in 3D printers using fused deposition modeling. Our approach offers a range of design parameters for controlling the properties of single strands and also of hair bundles. We further detail a list of post-processing techniques for refining the behavior and appearance of printed strands. We provide several examples of output, demonstrating the immediate feasibility of our approach using a low cost, commodity printer. Overall, this technique extends the capabilities of 3D printing in a new and interesting way, without requiring any new hardware.

Session 9A: Online Education

Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming BIBAFull-Text 599-608
  Philip J. Guo
One-on-one tutoring from a human expert is an effective way for novices to overcome learning barriers in complex domains such as computer programming. But there are usually far fewer experts than learners. To enable a single expert to help more learners at once, we built Codeopticon, an interface that enables a programming tutor to monitor and chat with dozens of learners in real time. Each learner codes in a workspace that consists of an editor, compiler, and visual debugger. The tutor sees a real-time view of each learner's actions on a dashboard, with each learner's workspace summarized in a tile. At a glance, the tutor can see how learners are editing and debugging their code, and what errors they are encountering. The dashboard automatically reshuffles tiles so that the most active learners are always in the tutor's main field of view. When the tutor sees that a particular learner needs help, they can open an embedded chat window to start a one-on-one conversation. A user study showed that 8 first-time Codeopticon users successfully tutored anonymous learners from 54 countries in a naturalistic online setting. On average, in a 30-minute session, each tutor monitored 226 learners, started 12 conversations, exchanged 47 chats, and helped 2.4 learners.
Foobaz: Variable Name Feedback for Student Code at Scale BIBAFull-Text 609-617
  Elena L. Glassman; Lyla Fischer; Jeremy Scott; Robert C. Miller
Current traditional feedback methods, such as hand-grading student code for substance and style, are labor intensive and do not scale. We created a user interface that addresses feedback at scale for a particular and important aspect of code quality: variable names. We built this user interface on top of an existing back-end that distinguishes variables by their behavior in the program. Therefore our interface not only allows teachers to comment on poor variable names, they can comment on names that mislead the reader about the variable's role in the program. We ran two user studies in which 10 teachers and 6 students created and received feedback, respectively. The interface helped teachers give personalized variable name feedback on thousands of student solutions from an edX introductory programming MOOC. In the second study, students composed solutions to the same programming assignments and immediately received personalized quizzes composed by teachers in the previous user study.
These Aren't the Commands You're Looking For: Addressing False Feedforward in Feature-Rich Software BIBAFull-Text 619-628
  Benjamin Lafreniere; Parmit K. Chilana; Adam Fourney; Michael A. Terry
The names, icons, and tooltips of commands in feature-rich software are an important source of guidance when locating and selecting amongst commands. Unfortunately, these cues can mislead users into believing that a command is appropriate for a given task, when another command would be more appropriate, resulting in wasted time and frustration. In this paper, we present command disambiguation techniques that inform the user of alternative commands before, during, and after an incorrect command has been executed. To inform the design of these techniques, we define categories of false-feedforward errors caused by misleading interface cues, and identify causes for each. Our techniques are the first designed explicitly to solve this problem in feature-rich software. A user study showed enthusiasm for the techniques, and revealed their potential to play a key role in learning of feature-rich software.

Session 9B: Pens, Mice and Sensor Strips

Looking through the Eye of the Mouse: A Simple Method for Measuring End-to-end Latency using an Optical Mouse BIBAFull-Text 629-636
  Géry Casiez; Stéphane Conversy; Matthieu Falce; Stéphane Huot; Nicolas Roussel
We present a simple method for measuring end-to-end latency in graphical user interfaces. The method works with most optical mice and allows accurate and real time latency measures up to 5 times per second. In addition, the technique allows easy insertion of probes at different places in the system I.e. mouse events listeners -- to investigate the sources of latency. After presenting the measurement method and our methodology, we detail the measures we performed on different systems, toolkits and applications. Results show that latency is affected by the operating system and system load. Substantial differences are found between C++/GLUT and C++/Qt or Java/Swing implementations, as well as between web browsers.
Joint 5D Pen Input for Light Field Displays BIBAFull-Text 637-647
  James Tompkin; Samuel Muff; James McCann; Hanspeter Pfister; Jan Kautz; Marc Alexa; Wojciech Matusik
Light field displays allow viewers to see view-dependent 3D content as if looking through a window; however, existing work on light field display interaction is limited. Yet, they have the potential to parallel 2D pen and touch screen systems, which present a joint input and display surface for natural interaction. We propose a 4D display and interaction space using a dual-purpose lenslet array, which combines light field display and light field pen sensing, and allows us to estimate the 3D position and 2D orientation of the pen. This method is simple, fast (150Hz), with position accuracy of 2-3mm and precision of 0.2-0.6mm from 0-350mm away from the lenslet array, and orientation accuracy of 2 degrees and precision of 0.2-0.3 degrees within a 45 degree field of view. Further, we 3D print the lenslet array with embedded baffles to reduce out-of-bounds cross-talk, and use an optical relay to allow interaction behind the focal plane. We demonstrate our joint display/sensing system with interactive light field painting.
SensorTape: Modular and Programmable 3D-Aware Dense Sensor Network on a Tape BIBAFull-Text 649-658
  Artem Dementyev; Hsin-Liu (Cindy) Kao; Joseph A. Paradiso
SensorTape is a modular and dense sensor network in a form factor of a tape. SensorTape is composed of interconnected and programmable sensor nodes on a flexible electronics substrate. Each node can sense its orientation with an inertial measurement unit, allowing deformation self-sensing of the whole tape. Also, nodes sense proximity using time-of-flight infrared. We developed network architecture to automatically determine the location of each sensor node, as SensorTape is cut and rejoined. Also, we made an intuitive graphical interface to program the tape. Our user study suggested that SensorTape enables users with different skill sets to intuitively create and program large sensor network arrays. We developed diverse applications ranging from wearables to home sensing, to show low deployment effort required by the user. We showed how SensorTape could be produced at scale using current technologies and we made a 2.3-meter long prototype.
FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications Using Single Shape-Sensing Strip BIBAFull-Text 659-663
  Chin-yu Chien; Rong-Hao Liang; Long-Fei Lin; Liwei Chan; Bing-Yu Chen
This paper presents FlexiBend, an easily installable shape-sensing strip that enables interactivity of multi-part, deformable fabrications. The flexible sensor strip is composed of a dense linear array of strain gauges, therefore it has shape sensing capability. After installation, FlexiBend can simultaneously sense user inputs in different parts of a fabrication or even capture the geometry of a deformable fabrication.

Closing Keynote Address

Machine Intelligence and Human Intelligence BIBAFull-Text 665
  Blaise Aguera y Arcas
There has been a stellar rise in computational power since 2006 in part thanks to GPUs, yet today, we are as an intelligent species essentially singular. There are of course some other brainy species, like chimpanzees, dolphins, crows and octopuses, but if anything they only emphasize our unique position on Earth -- as animals richly gifted with self-awareness, language, abstract thought, art, mathematical capability, science, technology and so on. Many of us have staked our entire self-concept on the idea that to be human is to have a mind, and that minds are the unique province of humans. For those of us who are not religious, this could be interpreted as the last bastion of dualism. Our economic, legal and ethical systems are also implicitly built around this idea. Now, we're well along the road to really understanding the fundamental principles of how a mind can be built, and Moore's Law will put brain-scale computing within reach this decade. (We need to put some asterisks next to Moore's Law, since we are already running up against certain limits in computational scale using our present-day approaches, but I'll stand behind the broader statement.) In this talk I will discuss the relationships between engineered neurally inspired systems and brains today, between humans and machines tomorrow, and how these relationships will alter user interfaces, software and technology.