HCI Bibliography Home | HCI Conferences | GamifIR Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
GamifIR Tables of Contents: 14

Proceedings of the 2014 International Workshop on Gamification for Information Retrieval

Fullname:GamifIR'14: First International Workshop on Gamification for Information Retrieval
Editors:Frank Hopfgartner; Gabriella Kazai; Udo Kruschwitz; Michael Meder
Location:Amsterdam, Netherlands
Standard No:ISBN: 978-1-4503-2892-0; ACM DL: Table of Contents; hcibib: GamifIR14
Links:Conference Website
Information reconstruction: unpicking the GamifIR call for papers BIBAFull-Text 1
  Richard A. Bartle
Gamification is a new and burgeoning field, with which Information Retrieval appears to have a particular affinity. In this talk, I consider what it is that Gamification offers Information Retrieval and at what potential cost. I do this by examining the Call for Papers for the workshop, attempting to tease out what Gamification is laying on the table that other approaches -- especially games themselves -- do not. Gamification is being consciously distanced from Serious Games, with only the "best bits" being used, but there appears to be no consensus as to why and (because it's so young a field and Information Retrieval is getting into it early) only a superficial understanding of what Gamification is. I believe that it's entirely possible that the uses of Gamification in Information Retrieval extend well beyond those envisaged.
   I conclude by arguing that although Game Design is an art form, Gamification is an application of Psychology. As successful Information Retrieval relies on an understanding of human motivation, it is therefore no surprise that the two are well suited to one another.
Creating Zombilingo, a game with a purpose for dependency syntax annotation BIBAFull-Text 2-6
  Karën Fort; Bruno Guillaume; Hadrien Chastant
This paper presents the design of Zombilingo, a Game With A Purpose (GWAP) that allows for the dependency syntax annotation of French corpora. The development will start mid-2014 and the game is to be made available by the end of the year. The created language resource will be freely and continuously available on the game Web site.
On the application of game mechanics in information retrieval BIBAFull-Text 7-11
  Luca Galli; Piero Fraternali; Alessandro Bozzon
The exponential growth of digital generated content in the form of audio, video and complex data structures calls for novel methods and tools able to cope with the limitation of automated analysis techniques.
   Gamification, the process of using game design methodologies and game mechanics to enhance traditional applications, is a promising tool that can help to increase the active involvement of humans in the Information Retrieval processes.
   This work contributes to the emerging research field of Gamification in Information Retrieval by providing an overview on: 1) the fundamental elements of a game; 2) the major game mechanics that have been applied in traditional games and gamification techniques; and 3) an overview of the possible adoption of such techniques in a typical IR scenario.
   The goal is to lay a path for the adoption of these new tools in IR systems, focusing on their application to the traditional building blocks of the query and content analysis processes.
The annotation-validation (AV) model: rewarding contribution using retrospective agreement BIBAFull-Text 12-16
  Jon Chamberlain
Evaluating contributions from users of systems with large datasets is a challenge across many domains, from task assessment in crowdsourcing to document relevance in information retrieval. This paper introduces a model for rewarding and evaluating users using retrospective validation, with only a small gold standard required to initiate the system. A simulation of the model shows that users are rewarded appropriately for high quality responses however analysis of data from an implementation of the model in a text annotation game indicates it may not be sophisticated enough to predict user performance.
The beauty contest revisited: measuring consensus rankings of relevance using a game BIBAFull-Text 17-21
  Christopher G. Harris
In this paper, we examine the Keynesian Beauty Contest, a well-known examination of rational agents used to explain the role of consensus predictions in decision making such as price fluctuations in equity markets. Using a game, we study the crowd's ability to judge relevance for both images and textual documents. In addition to asking participants to determine if a document is relevant, we also ask them to rank all choices. One group of participants (N=137) was asked to make judgments based on their own assessment while another group of participants (N = 137) was asked to make judgments based on their estimate of a consensus decision. In addition to measuring recall and precision, our game also uses rank-biased overlap (RBO) to compare each participant's ranked list with the overall consensus decision. Results show the group asked to make ranking decisions based on their estimate of consensus had significantly higher recall for judging relevance in text documents and significantly higher recall and precision when judging relevance for a set of images. We believe this has implications for the determination of consensus across multiple contexts.
People recognition using gamified ambiguous feedback BIBAFull-Text 22-26
  Markus Brenner; Navid Mirza; Ebroul Izquierdo
We present a semi-supervised approach to recognize faces or people while incorporating crowd-sourced and gamified feedback to iteratively improve recognition accuracy. Unlike traditional approaches which are often limited to explicit feedback, we model ambiguous feedback information that we implicitly gather through a crowd that plays a game. We devise a graph-based recognition approach that incorporates such ambiguous feedback to jointly recognize people across an entire dataset. Multiple experiments demonstrate the effectiveness of our gamified feedback approach.
Incentive design to mould online behavior: a game mechanics perspective BIBAFull-Text 27-32
  Dinesh Pothineni; Pratik Mishra; Aadil Rasheed; Deepak Sundararajan
Designing incentives to shape user behavior in line with system goals is a precarious feat. These goals can be diverse, ranging from user retention to streamlining content creation. Yet there is no playbook available in the art to be applied on such motivation problems. Pertaining to website rollout strategies, there exists a commonality in goals shared by businesses owners. We deployed an innovative incentive model in our enterprise social network knome to aid these goals. In this paper, we study how awarding points using 'Incentive Economy' strategy can mould online user behavior, in a way to meet the founding goals of a platform. We proposed few measures that can be adopted by other website owners to reach their behavior goals using game mechanics. With about 288,273 employees in TCS generating millions of interactions well over the span of 14 months, we extracted several interesting observations from knome. To our knowledge this is one of the largest studies of its nature done in enterprise settings, which in itself exposed unknown challenges. Our output suggests that incentives are effective in moulding user behaviour when coupled with right motivational strategies and targeted delivery, varying with personality types. Additional contributions include, an awarding strategy that can help websites to measure the effectiveness of individual game mechanics and incentives.
Gamification of private digital data archive management BIBAFull-Text 33-37
  Carlos Maltzahn; Arnav Jhala; Michael Mateas; Jim Whitehead
The super-exponential growth of digital data world-wide is matched by personal digital archives containing songs, ebooks, audio books, photos, movies, textual documents, and documents of other media types. For many types of media it is usually a lot easier to add items than to keep archives from falling into disarray and incurring data loss. The overhead of maintaining these personal archives frequently surpasses the time and patience their owners are willing to dedicate to this important task. The promise of gamification in this context is to significantly extend the willingness to maintain personal archives by enhancing the experience of personal archive management.
   In this paper we focus on a subcategory of personal archives which we call private archives. These are archives that for a variety of reasons the owner does not want to make available online and which consequently limits archive maintenance to an individual activity and does not allow any form of crowdsourcing out of fear for unwanted information leaks. As an example of private digital archive maintenance gamification we describe InfoGarden, a casual game that turns document tagging into an individual activity of (metaphorically) weeding a garden and protecting plants from gophers and includes a reward system that encourages orthogonal tag usage. The paper concludes with lessons learned and summarizes remaining challenges.
PageFetch 2: gamification the sequel BIBAFull-Text 38-41
  Leif Azzopardi; Martin Bevc; Andrew Gardner; David Maxwell; Abdullah Razzouk
This demo poster presents PageFetch 2, the sequel of PageFetch, Fu-Finder and PageHunt. The game is simple. A player is shown a Webpage. The player must then enter a query that they believe will retrieve the displayed page. The shorter the query and the higher the page appears in the returned rankings, the more points the player receives. Players have three minutes to find as many of the displayed pages as possible. What makes this iteration different is the inclusion of several additional gamification features other than just points and an updated scoring system. We include leaderboards, badges and an avatar component -- all of which are designed to help and motivate players. The framework is also publicly available for download, and can be customised to create new search games -- or even to evaluate different search engines and document collections. This demo is a preliminary report of the progress we have made to date. At the workshop, we hope to solicit feedback on how we can improve the game and its gameplay.
Enhancing collaborative search systems engagement through gamification BIBAFull-Text 42-45
  Juan M. Fernández-Luna; Juan F. Huete; Humberto Rodríguez-Avila; Julio C. Rodríguez-Cano
Collaborative search systems require to make sense of the information to achieve shared goals and reduce additional costs. These systems include cognitive load and the cost to coordinate various events and participants. By applying game mechanics and dynamics to these collaborative systems we can intensify seeker engagement with this type of applications and its collaborative tasks. To the best of our knowledge the use of game design elements in non-gaming contexts has been so far discussed in a few research contexts, without ties to existing literature on Collaborative Information Seeking. This paper presents a set of elements to gamify collaborative search systems.
The game of search: what is the fun in that? BIBAFull-Text 46-48
  Mark Shovman
Gamifying the core IR task, i.e. web search, is a challenging, but potentially rewarding task. We present several approaches to tackling it, going beyond the PBL (points-badges-leaderboards) triad: analysing actual and potential hedonic elements, and suggesting appropriate ludic mechanisms.
Studying user browsing behavior through gamified search tasks BIBAFull-Text 49-52
  Jiyin He; Marc Bron; Leif Azzopardi; Arjen de Vries
Typical crowdsourcing tasks ask workers to label images or make relevance judgements, as a low cost alternative to lab based user studies. More recently, gamification has been employed as a way to make these tasks more appealing and so users play, rather than work. One observation is that differences in task design and incentives elicits different player behavior. In this paper we discuss a new type of task, where we aim at eliciting player behavior that resembles user behavior when performing a search task. Care should be taken in the design of a gamified version of such a task to allow players to complete tasks with a limited amount of effort and time, without changing the behavior to be studied. We discuss the motivation of the abstractions and design choices we have made in achieving this goal. We then analyze whether and how these abstractions and design choices influence our observations of player behaviors.
Crowd-powered experts: helping surgeons interpret breast cancer images BIBAFull-Text 53-56
  Carsten Eickhoff
Crowdsourcing is often applied for the task of replacing the scarce or expensive labour of experts with that of untrained workers. In this paper, we argue, that this objective might not always be desirable, but that we should instead aim at leveraging the considerable work force of the crowd in order to support the highly trained expert. In this paper, we demonstrate this different paradigm on the example of detecting malignant breast cancer in medical images. We compare the effectiveness and efficiency of experts to that of crowd workers, finding significantly better performance at greater cost. In a second series of experiments, we show how the comparably cheap results produced by crowdsourcing workers can serve to make experts more efficient AND more effective at the same time.
PictureSort: gamification of image ranking BIBAFull-Text 57-60
  Mathias Lux; Mario Guggenberger; Michael Riegler
Human computation is a very powerful tool for solving tasks that cannot be solved by computers efficiently. One such problem is ranking images upon their relevance for a semantic query or upon how well they depict a semantic concept. In this paper we investigate a method to leverage human computation in a divide-and-conquer approach to create precise ranking models. We discuss the basic technique, our prototype client, its adoption to a gamification approach, and present the results of a study with the prototype. Results from the study indicate that with our method the ranking aggregated from the user input converges fast to an optimal ranking.
Moody closet: exploring intriguing new views on wardrobe recommendation BIBAFull-Text 61-62
  Bojana Dumeljic; Martha Larson; Alessandro Bozzon
This paper introduces Moody Closet, a mobile application for the management of a personal wardrobe with a personalized outfit recommender. To provide incentive for the users to add content and express their preferences, the system provides an easy and enjoyable interaction, which delivers new perspectives on their closets. In particular, we focus on the mood of the wearer, which is considered to be an intriguing trigger capable of prompting the contribution of information needed to feed a recommendation system. An exploratory study with a small set of users provides an initial demonstration that the concept has the potential to fascinate users and motivate them to contribute. We demonstrate a working prototype which showcases the addition of content and the triggers provided to motivate this process.