HCI Bibliography Home | HCI Conferences | IIiX Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
IIiX Tables of Contents: 0608101214

Proceedings of the 2012 Symposium on Information Interaction in Context

Fullname:Proceedings of the 4th Symposium on Information Interaction in Context
Editors:Norbert Fuhr; Jaap Kamps; Wessel Kraaij
Location:Nijmegen, Netherlands
Dates:2012-Aug-21 to 2012-Aug-24
Standard No:ISBN: 978-1-4503-1282-0; ACM DL: Table of Contents; hcibib: IIiX12
Links:Conference Website
  1. Keynote presentations
  2. Finding books
  3. Expression and understanding in interaction
  4. Search user interface design
  5. Queries in context
  6. Web search behaviour
  7. Wikipedia and cultural heritage search behaviour session
  8. Representations, visualizations and behaviour session
  9. Information seeking in specific applications
  10. Poster session
  11. Doctoral consortium session

Keynote presentations

Citations and references as keys to relevance ranking in interactive IR BIBAFull-Text 1
  Peter Ingwersen
According to the principle of Polyrepresentation (Ingwersen & Järvelin, 2005; Ingwersen, 2012) bibliographic references in scientific documents as well as citations to documents have the potential of serving as useful features for re-ranking of retrieved documents. References (and thus citations) can be seen as footprints of information interaction, because of the behavioral conventions built in to the scientific communication and publication process. They are manifestations of degrees of utility of methods, results and ideas made earlier on by other scientists. The use of references in IR has been demonstrated to improve retrieval performance (Skov et al. 2008), whereas the number of citations has not provided similar improvements.
Cognitive consequences of search BIBAFull-Text 2
  Diane Kelly
For some time now, the general goal of information retrieval (IR) has been to present a user with an optimally ranked set of results as quickly as possible. At first glance, things seem to be working well: users often find what they need on the first search results page, they do not have to create their own queries or read through multiple pages of text, and soon they may not even have to think of their own information needs. Researchers document success by showing reductions in time and amount of interaction, and increased user satisfaction, but do these measures really allow researchers to understand the impact of search?
What does it mean to be literate in the age of Google? BIBAFull-Text 3
  Daniel Russell
What does it mean to be literate at a time when you can search over billions of texts in less than 300 milliseconds? Although you might think that "literacy" is one of the great constants that transcends the ages, the skills of a literate person have changed substantially over time as texts and technology allow for new kinds of reading and understanding. Knowing how to read is just the beginning of it -- knowing how to frame a question, pose a query, how to interpret the texts that you find, understand the information in context, how to organize and use the information you discover, how to understand your metacognition -- these are all critical parts of being literate as well. In this talk I'll review what literacy is today, in the age of Google, and show how some very surprising and unexpected skills will turn out to be critical in the years ahead.

Finding books

Examining the effect of task stage and topic knowledge on searcher interaction with a "digital bookstore" BIBAFull-Text 4-11
  Nils Pharo; Ragnar Nordlie
This paper reports some results from the experiment of the 2010 INEX interactive track. The experiment was designed to let searchers simulate being at two distinct stages of a work task process. Data were also collected on the test participants' topic knowledge. We have performed statistical analysis of the collected data to study differences with respect to relevance judgments and use of different types of metadata, at the different stages and for users with high and low topic knowledge.
In search of a good novel, neither reading activity nor querying matter, but examining search results does BIBAFull-Text 12-20
  Suvi Oksanen; Pertti Vakkari
Borrowing novels is a major activity in public libraries. However, the interest in developing tools for fiction searching and analyzing the use of these tools is minor. It is studied how tools provided by an enriched public library catalogue are used to access novels to read. 58 users searched for interesting novels to read in a simulated situation where they had only a vague idea of what they would like to read. Data consist of search logs, pre and post search questionnaires and observations. For analyzing associations between novel reading activity, search variables and search success Pearson correlation coefficients were calculated. Based on this information, path models were built for predicting search success, i.e. the interest ratings of the novels found. Investing effort on examining results improves search success, i.e. finding interesting novels, whereas effort in querying has no bearing on it. Novel reading activity was not associated with search process and effort and success variables observed. The results suggest, that in designing systems for fiction retrieval, enriching result presentation with more detailed book information would benefit users in identifying good novels.
Exploring evaluation criteria of social navigational tools on social media: a case study of aNobii BIBAFull-Text 21-26
  Muh-Chyun Tang; Pei-Hang Ting; Yi-Jin Sie
A user study of aNobii was conducted to compare its three book-finding tools: author search, browsing friends' bookshelves and browsing similar bookshelves. The construct of "social navigation" was identified as a useful theoretical framework to discuss various modes of information access on social media.
   A within-subject experimental design was adopted where all forty regular aNobii users searched alternately with the three book-finding tools. Several novel evaluation measures were designed to explore the potential benefits these tools might bring to the users. Other than the self-report user experience and search result measures, the "consideration set" model was used as a novel framework for navigational effectiveness.
   Some major findings are as follows. While the author search function was shown to be the most efficient, browsing friends' bookshelves was shown to generate more interesting and informative browsing experience. Three evaluative dimensions were derived from our study: search experience, search efficiency, and search result quality. The disagreement of these measures shows a need for a multi-faceted evaluative framework for these exploration-based navigational tools.

Expression and understanding in interaction

Phonological working memory impacts on information searching: an investigation of dyslexia BIBAFull-Text 27-34
  A. MacFarlane; A. Albrair; C. R. Marshall; G. Buchanan
A key aspect of searching is the ability of users to absorb information from documents read in order to resolve their task. One group of users who have problems with reading are dyslexic users, who due to underlying cognitive impairments in phonological processing and working memory, tend to read more slowly and make reading errors. The purpose of this study is to examine the impact of the dyslexia cognitive profile on information searching. Searches were logged for 8 dyslexic and 8 non-dyslexic university students, in order to examine the differences in searching behavior between the two groups. A set of literacy and phonological working memory tasks were also completed, in order to investigate the relationship between these cognitive variables and searching behavior. Results show that there is a significant difference between the two groups on the number of documents being judged irrelevant, and that this cannot be explained by a topic effect. Instead, the number of documents judged irrelevant is significantly correlated with a measure of working memory. This key result provides the research community the first real insight into impact of impaired working memory on information searching.
Human question answering performance using an interactive document retrieval system BIBAFull-Text 35-44
  Mark D. Smucker; James Allan; Blagovest Dachev
Every day, people answer their questions by using document retrieval systems. Compared to document retrieval systems, question answering (QA) systems aim to speed the rate at which users find answers by retrieving answers rather than documents. To better understand how document retrieval systems compare to QA systems, we measured the performance of humans using an interactive document retrieval system to answer questions. We first measured the ability of users to answer their questions using an interactive document retrieval system, and then compared the users' performance with the document retrieval system to question answering systems. We found that while users can successfully answer their questions using a document retrieval system, question answering systems have the potential to significantly increase the rate at which users find answers.
On sociocultural aspects of user elicitation BIBAFull-Text 45-51
  Ying-Hsang Liu; Mei-Mei Wu
Elicitation (i.e., question-asking) is complex human linguistic behavior that can provide insights into people's knowledge state, intention and action for user modeling in information retrieval (IR) research. This paper reports a comparative study of user elicitation behaviors by Chinese and English speakers in professional settings of mediated search, with particular reference to sociocultural aspects of human communication. More specifically, this study investigates user elicitation behavior from real-life interactional cases, with the goal of exploring whether there are differences between Chinese and English speaking users in their elicitation behavior. To reveal the sociocultural influences in user-intermediary interactions, corpus data is analyzed in terms of the identification and counting of the purposes and communicative functions. Our findings revealed significant differences in the use of elicitation purposes and communicative functions by Chinese and English users. Chinese users were more concerned about the technical aspects of searching activities, with little reference to the cognitive aspects of their information problems. Importantly, the differences in communicative functions (i.e., the intended meaning) suggested potential sociocultural influences on micro-level information seeking. The findings are discussed in view of IR system design supporting query formulation, user modeling in interactive IR and conceptual modeling in information behavior research.

Search user interface design

The future is in the past: designing for exploratory search BIBAFull-Text 52-61
  Gene Golovchinsky; Abdigani Diriye; Tony Dunnigan
Exploratory search activities tend to span multiple sessions and involve finding, analyzing and evaluating information found through many queries. Typical search systems, on the other hand, are designed to support single query, precision-oriented search tasks. We describe a search interface and system design of a multi-session exploratory search system, discuss design challenges encountered, and chronicle the evolution of our design. Our design describes novel displays for visualizing retrieval history information, and introduces ambient displays and persuasive elements to interactive information retrieval.
A permeable expert search strategy approach to multimodal retrieval BIBAFull-Text 62-71
  David Zellhöfer
This paper presents an interactive multimodal retrieval system featuring multiple search strategies. In contrast to the system-centric perspective often found in multimedia retrieval, we follow a more user-centered approach considering the search as an interactive process. To assist in this process, the discussed system supports directed and exploratory search as well as faceted navigation and a transition between these information seeking strategies.
   In order to integrate these strategies, a consistent retrieval and interaction model based on the principle of polyrepresentation is developed. To complete the functionality, a preference-based mechanism for graded relevance feedback is presented that overcomes limitations of binary as well as total order-based approaches. To improve the learnability of the system and to give users back the feeling of control over the search process, various visualizations are offered that open paths of communication between the system and the user in order to bridge the gap between the system's notion of the information need and the one of the actual user.
Information vs interaction: examining different interaction models over consistent metadata BIBAFull-Text 72-81
  Kingsley Hughes-Morgan; Max L. Wilson
In the quest to develop better and more useful search systems, many novel search user interface features have been developed, such as relevance feedback, clusters, tag clouds, facets, and so on. Yet all of these novel 'interactions' have required novel forms of 'information', or metadata, to make them work. Consequently, we do not know whether users have been benefiting from better interaction or simply richer forms of metadata, or both. In this research, we aimed to show that better interaction can be provided, regardless of whether we have access to, or the ability to generate, richer forms of metadata. Using only search engine query suggestions as a consistent form of metadata, we built interface conditions for three common interaction models for search: query suggestions (our baseline), hierarchical browsing, and faceted filtering. Our results showed that, despite interacting with the same underlying metadata, users experienced significant performance gains with different forms of interaction. These findings have implications for search user interface designers, who are often working with fixed metadata or within limited budgets. Our future work will focus on complementing these findings by recreating the same interaction with different forms of metadata, such that we can then compare the performance gain separately provided by both information and interaction.

Queries in context

An analysis of free-text queries for a multi-field web form BIBAFull-Text 82-89
  Kien Tjin-Kam-Jet; Dolf Trieschnigg; Djoerd Hiemstra
We report how users interact with an experimental system that transforms single-field textual input into a multi-field query for an existing travel planner system. The experimental system was made publicly available and we collected over 30,000 queries from almost 12,000 users. From the free-text query log, we examined how users formulated structured information needs into free-text queries. The query log analysis shows that there is great variety in query formulation, over 400 query templates were found that occurred at least 4 times. Furthermore, with over 100 respondents to our questionnaire, we provide both quantitative and qualitative evidence indicating that end-users significantly prefer a single field interface over a multi-field interface when performing structured search.
Entertainment on the go: finding things to do and see while visiting distributed events BIBAFull-Text 90-99
  Richard Schaller; Morgan Harvey; David Elsweiler
Distributed events are collections of single events taking place within a small geographical area at approximately the same time, normally related to one given topic e.g. music, film, arts etc. There are usually a large number of events on offer and the times in which they can be visited are heavily constrained. Therefore the information seeking task of choosing the events to visit and in which order can be very difficult.
   In this paper we investigate, via 2 large-scale naturalistic studies (n=391 and n=740), how mobile applications can be designed to assist users in this task and how such applications are used. We present an application that allows users to search and browse the events on offer in a number of different ways including via personalised event recommendations. Logs were collected of user interactions with the system. The results of this log analysis in combination with 2 surveys show some surprising usage patterns and point to how such applications can better serve users' needs.
Generating queries from user-selected text BIBAFull-Text 100-109
  Chia-Jung Lee; W. Bruce Croft
People browsing the web or reading a document may see text passages that describe a topic of interest, and want to know more about it by searching. Manually formulating a query from that text can be difficult, however, and an effective search is not guaranteed. In this paper, to address this scenario, we propose a learning-based approach which generates effective queries from the content of an arbitrary user-selected text passage. Specifically, the approach extracts and selects representative chunks (noun phrases or named entities) from the content (a text passage) using a rich set of features. We carry out experiments showing that the selected chunks can be effectively used to generate queries both in a TREC environment, where weights and query structure can be directly incorporated, and with a "black-box" web search engine, where query structure is more limited.

Web search behaviour

Ordinary search engine users assessing difficulty, effort, and outcome for simple and complex search tasks BIBAFull-Text 110-119
  Georg Singer; Ulrich Norbisrath; Dirk Lewandowski
Search engines are the preferred tools for finding information on the Web. They are advancing to be the common helpers to answer any of our search needs. We use them to carry out simple look-up tasks and also to work on rather time consuming and more complex search tasks. Yet, we do not know very much about the user performance while carrying out those tasks -- especially not for ordinary users. The aim of this study was to get more insight into whether Web users manage to assess difficulty, time effort, query effort, and task outcome of search tasks, and if their judging performance relates to task complexity. Our study was conducted with a systematically selected sample of 56 people with a wide demographic background. They carried out a set of 12 search tasks with commercial Web search engines in a laboratory environment. The results confirm that it is hard for normal Web users to judge the difficulty and effort to carry out complex search tasks. The judgments are more reliable for simple tasks than for complex ones. Task complexity is an indicator for judging performance.
Grieving online: the use of search engines in times of grief and bereavement BIBAFull-Text 120-128
  Ian Ruthven
In this paper, we study the information goals in searches on the topic of grieving and bereavement. Using log analyses and content analyses we present a categorization system of grief and bereavement-related queries submitted to a major search engine, reflecting the variety of information needs that occur during a period of bereavement. We also present patterns of interaction during the query formulation stage of these searches and results on the success of searches on the topic of grieving and bereavement. Using linguistic style analyses we compare the content of grief-related queries showing differences in content that may be used to classify grief-related queries to help provide tailored support for different types of query.
Supporting children's web search in school environments BIBAFull-Text 129-137
  Carsten Eickhoff; Pieter Dekker; Arjen P. de Vries
Nowadays, the Internet represents a ubiquitous source of information and communication. Its central role in everyday life is reflected in the curricula of modern schools. Already in early grades, children are encouraged to search for information on-line. However, the way in which they interact with state-of-the-art search interfaces and how they explore and interpret the presented information, differs greatly from adult user behaviour.
   This work describes a qualitative user study in which the Web search behaviour of Dutch elementary school children was observed and classified into roles motivated by prior research in cognitive science. Building on the findings of this survey, we propose an automatic method of identifying struggling searchers in order to enable teaching personnel to provide appropriate and targeted guidance where needed.

Wikipedia and cultural heritage search behaviour session

Intention and task context connected with session in a cultural heritage collection BIBAFull-Text 138-144
  Jonas Fransson
In this paper two methods are used, analysis of web access logs together with a web survey, to connect sessions with task context and intention. The dual methods approach connects information seeking behavior with actual IR-interactions to get a more holistic view of the information seeking and retrieval (IS&R) process. The studied collection is a cultural heritage web site containing digitalized material and artist information. Based on different navigation strategies task context and intention was related to session length and arrival level in the site. Some statistically significant relationships were found between the factors. Users in a work context more often access the site by direct navigation for looking up facts. Users in a hobby or leisure context more often access the site by links or topical searches in search engines and they arrive further down in the site structure and their sessions are shorter. How and to what extent survey data and log data can be combined is discussed.
Looking for genre: the use of structural features during search tasks with Wikipedia BIBAFull-Text 145-154
  Malcolm Clark; Ian Ruthven; Patrik O'Brian Holt; Dawei Song
This paper reports on our task-based observational, logged, questionnaire study and analysis of ocular behavior pertaining to the interaction of structural features of text in Wikipedia using eye tracking. We set natural and realistic tasks searching Wikipedia online focusing on examining which features and strategies (skimming or scanning) were the most important for the participants to complete their tasks. Our research, carried out on a group of 30 participants, highlighted their interactions with the structural areas within Wikipedia articles, the visual cues and features perceived during the searching of the Wiki text. We collected questionnaire and ocular behavior (fixation metrics) data to highlight the ways in which people view the features in the articles. We found that our participants' extensively interacted with layout features, such as tables, titles, bullet lists, contents lists, information boxes, and references. The eye tracking results showed that participants used the format and layout features and they also highlighted them as important. They were able to navigate to useful information consistently, and they were an effective means of locating relevant information for the completion of their tasks with some success. This work presents results which contribute to the long-term goals of studying the features for genre and theoretical perception research.
Surfin' Wikipedia: an analysis of the Wikipedia (non-random) surfer's behavior from aggregate access data BIBAFull-Text 155-163
  Karl Gyllstrom; Marie-Francine Moens
Does the reason a user visits a Wikipedia page influence that user's subsequent browsing behavior on Wikipedia? We address this question using aggregate Wikipedia page access data. We conduct: (1) a comparison of browsing behaviors between serendipitous and directed information seekers; and (2) how topic/category influences users' migration from page to page. Our findings indicate that surfer behavior and topic are potentially influential factors, which weakens the random surfer model underlying link-based algorithms, and could affect how web designers should design sites to best meet the diversity of information seeking behaviors.

Representations, visualizations and behaviour session

On the consistency and features of image similarity BIBAFull-Text 164-173
  Pierre Tirilly; Xiangming Mu; Chunsheng Huang; Iris Xie; Wooseob Jeong; Jin Zhang
Image indexing and retrieval systems mostly rely on the computation of similarity measures between images. This notion is ill-defined, generally based on simplistic assumptions that do not fit the actual context of use of image retrieval systems. This paper addresses two fundamental issues related to image similarity: checking whether the degree of similarity between two images is perceived consistently by different users and establishing the elements of the images on which users base their similarity judgment. A study is set up, in which human subjects have been asked to assess the degree of the pairwise similarity of images and describe the features on which they base their judgments. The quantitative analysis of the similarity scores reported by the subjects shows that users reach a certain consensus on similarity assessment. From the qualitative analysis of the transcripts of the records of the experiments, a list of the features used by the subjects to assess image similarity is built. From this, a new model of image description emerges. As compared to existing models, it is more realistic, free of preconceptions and more suited to the task of similarity computation. These results are discussed from the perspectives of psychology and computer science.
Preliminary experiments using subjective logic for the polyrepresentation of information needs BIBAFull-Text 174-183
  Christina Lioma; Birger Larsen; Peter Ingwersen
According to the principle of polyrepresentation, retrieval accuracy may improve through the combination of multiple and diverse information object representations about e.g. the context of the user, the information sought, or the retrieval system [9, 10]. Recently, the principle of polyrepresentation was mathematically expressed using subjective logic [12], where the potential suitability of each representation for improving retrieval performance was formalised through degrees of belief and uncertainty [15]. No experimental evidence or practical application has so far validated this model.
   We extend the work of Lioma et al. (2010) [15], by providing a practical application and analysis of the model. We show how to map the abstract notions of belief and uncertainty to real-life evidence drawn from a retrieval dataset. We also show how to estimate two different types of polyrepresentation assuming either (a) independence or (b) dependence between the information objects that are combined. We focus on the polyrepresentation of different types of context relating to user information needs (i.e. work task, user background knowledge, ideal answer) and show that the subjective logic model can predict their optimal combination prior and independently to the retrieval process.
What does time spent on searching indicate? BIBAFull-Text 184-193
  Pia Borlund; Sabine Dreier; Katriina Byström
In this paper, we report a comparative study on what users' time spent on searching for information is an indication of. Time spent is commonly interpreted as an implicit measure of interest, but might indeed describe other circumstances of the information retrieval (IR) interaction. This phenomenon of time spent is interesting from an IR evaluation point of view with reference to how time spent is to be interpreted. A comparison of time spent between a semi-lab interactive IR (IIR) study using simulated work task situations and a naturalistic IIR study is presented. The findings of this comparison are further related to a study on information searching and seeking in the real work environment that provides a resonance board for the reported IIR studies. The main conclusion is that time spent searching depends not only on interest, but also on circumstances such as prior knowledge and external requirements.
Visual interactive failure analysis: supporting users in information retrieval evaluation BIBAFull-Text 194-203
  Marco Angelini; Nicola Ferro; Gianmaria Silvello; Giuseppe Santucci
Measuring is a key to scientific progress. This is particularly true for research concerning complex systems, whether natural or human-built. Multilingual and multimedia information access systems, such as search engines, are increasingly complex: they need to satisfy diverse user needs and support challenging tasks. Their development calls for proper evaluation methodologies to ensure that they meet the expected user requirements and provide the desired effectiveness. In this context, failure analysis is crucial to understand the behaviour of complex systems. Unfortunately, this is an especially challenging activity, requiring vast amounts of human effort to inspect query-by-query the output of a system in order to understand what went well or bad. It is therefore fundamental to provide automated tools to examine system behaviour, both visually and analytically. Moreover, once you understand the reason behind a failure, you still need to conduct a "what-if" analysis to understand what among the different possible solutions is most promising and effective before actually starting to modify your system. This paper provides an analytical model for examining performances of IR systems, based on the discounted cumulative gain family of metrics, and visualization for interacting and exploring the performances of the system under examination. Moreover, we propose machine learning approach to learn the ranking model of the examined system in order to be able to conduct a "what-if" analysis and visually explore what can happen if you adopt a given solution before having to actually implement it.

Information seeking in specific applications

Task complexity and information searching in administrative tasks revisited BIBAFull-Text 204-213
  Miamaria Saastamoinen; Sanna Kumpulainen; Kalervo Järvelin
In task-based information searching, the task at hand is a central factor affecting information search. Task complexity, in particular, has been discovered to affect searching. In the present study, we shadowed the tasks of seven people working in city administration. The data consist of shadowing field notes, voice recordings, photographs and forms. We study, how task complexity affects information searching and information resource use. Task complexity was defined through the task performer's own experience (perceived task complexity) and her estimates of her a priori knowledge concerning the task. We analyzed the data both qualitatively and quantitatively, focusing on the links between task complexity and the use of information resources, information searching and problems encountered. We found that task complexity has a central but ambiguous relationship to task performance. The clearest differences were found between simple and complex tasks. In addition, perceived task complexity seems to affect the ways of performing the task more than a priori knowledge. The more complex a task is perceived, the more searches are performed and the more they concentrate on networked resources instead of information systems provided by the organization (SPOs). The use of resources on the task performer's PC and the SPOs decreases when complexity increases. In proportion, the use of networked resources and communication resources increases. The total number of information resources used is somewhat greater in complex and semi-complex tasks than in simple tasks; and each resource is used for a longer time on average. Our study shows that task context and especially task complexity seems to affect information searching and the selection of sources.
Readers' search strategies for accessing books in public libraries BIBAFull-Text 214-223
  Anna Mikkonen; Pertti Vakkari
The aim of this study was to survey typical search tactics used by readers in public libraries. We analyzed whether demographic factors, the frequency of library visits, reading activity and the type of books read were associated with major search tactics. The data consisted of 1000 respondents of mail questionnaire collected in 2010 as a stratified random sample of the Finnish population aged 15 to 79 years. The most often used search tactics for accessing books in public libraries were known book or author search together with browsing search. The use of known item search strategy was associated with high educational level and the amount of non-fiction books read. The use of browsing search strategy was associated with the frequency of library visits, gender and the amount of fiction books read.
Scientists' preferences for bioinformatics tools: the selection of information retrieval systems BIBAFull-Text 224-233
  Joan C. Bartlett; Yusuke Ishimura; Lorie A. Kloda
In this paper, we present the findings of a survey of scientists regarding the characteristics they rate as important in their selection of bioinformatics tools. A total of 223 biologists completed a web-based questionnaire in which they assessed 39 individual characteristics of bioinformatics tools, independently rating the importance of each one in their selection of bioinformatics tools. The most highly rated characteristic was whether a tool actually accomplished what a participant needed it to do. Other highly rated characteristics included the ease of use of a tool, online availability, little or no cost, the ability to handle large data sets, and compatibility with other tools. Rated as non-important were characteristics relating to an imposed choice, such as a tool already having been purchased. Differences in ratings were found between those whose work was primarily focused on laboratory biology, and those whose work was primarily computer focused. The findings provide a set of characteristics by which bioinformatics tools can be annotated to facilitate the selection of tools. The findings also highlight the need to consider the requirements of different user groups in the design and development of information systems, rather than assuming that one size fits all.

Poster session

Studying information interaction in context: some lessons for traffic experiments BIBAFull-Text 234-237
  Martin Colbert; Adam Oliver
One way of studying information interaction in context is to conduct traffic experiments. We recently compared different versions of a small website with low traffic volumes to demonstrate some hypothesised effects on information interaction. In this paper, we describe the methodological lessons we learnt when conducting these experiments. The lessons concern: i) implementing versions of the site in a way that supports parallel testing; and ii) designing the web site to operationalise information quality variables (persuasiveness, credibility etc).
Model for simulating result document browsing in focused retrieval BIBAFull-Text 238-241
  Paavo Arvola; Johanna Vainio; Marko Junkkari; Jaana Kekäläinen
A search process is a ternary relationship between the user, the retrieval system and the user interface. A focused retrieval system aims at retrieving the most relevant parts within a relevant document. In focused retrieval the user interface may show not only the relevant documents but also identify the most relevant parts within a document. In document browsing a user may apply several strategies, related to the order of browsing, tolerance to irrelevance and the amount of required information. A model for result document browsing in focused retrieval is introduced. Further, its applicability is illustrated through a simulated experiment.
PatentLight: a patent search application BIBAFull-Text 242-245
  Silvia Calegari; Emanuele Panzeri; Gabriella Pasi
Patent retrieval is a complex challenging task; despite of the numerous patent search applications, more flexible solutions are needed to help users to easily identify relevant patents. This paper presents the PatentLight search tool that offers novel and flexible functionalities to help users looking for relevant patents represented in the XML format.
Modelling contexts for information interaction as "360°" user journeys: an initial illustration with reference to pregnant women quitting smoking BIBAFull-Text 246-249
  Nalini Edwards; Martin Colbert
In this paper, we illustrate use of the "360°" framework and notation to model persuasive user journeys, with a view to supporting the development of multi-channel smoking cessation services, applications and information. The illustration concerns a pregnant woman attempting to quit smoking.
Visual metaphors to model metacognitive strategies that support memory during the process of refinding information BIBAFull-Text 250-253
  Leanne Bowler; Eleanor Mattern
This paper reports on a study that models the metacognitive thinking of users in relation to memory and refinding information. Twenty-seven participants, in five separate groups ranging in age from 13 to early 30's, sketched visual metaphors representing strategies and interventions that the participants thought would remind them to remember before information was lost, in order to better relocate information. Nine themes emerged: embeddedness, fear and anxiety, interruptions, messiness and discomfort, locked doors and barriers, proximity and adjacency, signs and tattoos, scripts, and finally, the voice. This study started from the premise that design should begin with the user's metaphor as a way to describe the user's mind and ways of thinking and end with the designer mapping the metaphor to the artifact. The long term goal of this work is to move from ideation to implementation, using the users' metaphors of the mind as a basis for the design of information environments that scaffold metacognition during the search process.
Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity BIBAFull-Text 254-257
  Wan-Ching Wu; Diane Kelly; Ashlee Edwards; Jaime Arguello
One of the most challenging aspects of designing an interactive information retrieval (IIR) study is the development of search tasks. In this paper, we present preliminary results of a study designed to evaluate a set of search tasks that were developed for use in IIR studies. We created 20 search tasks using five levels of cognitive complexity and four domains, and conducted a laboratory evaluation of these tasks with 48 undergraduate subjects. We describe preliminary results from an analysis of data from 24 subjects for 10 search tasks. Initial results show that, in general, as cognitive complexity increased, subjects issued more queries, clicked on more search results, viewed more URLs and took more time to complete the task. Subjects' expected and experienced difficulty ratings of tasks generally increased as cognitive complexity increased with some exceptions. When subjects were asked to rank tasks according to difficulty and engagement, tasks with higher cognitive complexity were rated as more difficult than tasks with lower cognitive complexity, but not necessarily as more engaging. These preliminary results suggest that behaviors and ratings are fairly consistent with the differences one might expect among the search tasks and provide initial evidence of the usefulness of these tasks in IIR studies.
Towards a model of collaborative information retrieval in tourism BIBAFull-Text 258-261
  Abu Shamim Mohammad Arif; Jia Tina Du; Ivan Lee
The research area of collaborative information retrieval (CIR) has received increasing interest from interactive information retrieval researchers in recent years. Various studies on CIR have been conducted within different contexts and illuminated some key aspects of CIR. However, the research area of CIR for tourism is still in its early stages. This paper reports on our ongoing research into the development of a collaborative tourism information retrieval model that combines the processes of tourists' collaboration and information retrieval. Based on prior research and our survey results, a preliminary theoretical model of CIR in tourism domain was developed. The model provides insights for our future design of a collaborative tourism information retrieval system.
A preliminary study using PageFetch to examine the searching ability of children and adults BIBAFull-Text 262-265
  James Purvis; Leif Azzopardi
Evaluating the children's information seeking behaviors and information retrieval abilities poses a number of difficult challenges for researchers to overcome. One of the main problems is engaging children to undertake search tasks so that their abilities at retrieving relevant information can be assessed. In this poster paper, we outline PageFetch, which is an Information Retrieval based game designed to engage information seekers of all ages, but particular, children, to play and thus provide valuable data to assess and compare their search abilities to other age groups. We also report the results from an initial pilot study using PageFetch where over 140 participants played approximately 1500 games.
   While, previous research has shown that children do not perform as well as adults, our finding suggest that given modern search engines, children (or more specifically teenagers) are more than capable of finding specified pages -- and in fact for topics that they are more likely to be interested in, they often out perform adults. Since, these findings are very preliminary, they do raise a number of questions about the quality of modern search engines and the search efficacy of younger searchers. This work motivates the undertaking of secondary and larger study that examines on a year by year basis how search skills develop and improve from childhood to adulthood.
Graphical representation and similarity measurement of relevance judgments on the web BIBAFull-Text 266-269
  Panos Balatsoukas; Ian Ruthven
The purpose of this paper is to present a method for the graphical representation and similarity measurement of relevance judgments on the web. In order to address this objective a Latent Semantic Indexing technique was used. The findings suggest that the proposed method could help researchers in information seeking and retrieval to make methodological decisions about their data, such as the selection of specific subsets of relevance judgments for further examination, the recording of dissimilarities between judgments, or, the identification of possible cognitive shifts and abnormalities in relevance judgment behavior during web searching.
Practical considerations when filtering documents BIBAFull-Text 270-273
  Desmond Elliott; Leif Azzopardi
Implementing, configuring, and running an information filtering system in a practical setting is a difficult and challenging problem. This is due to variety and configuration of available system components along with additional factors such as topic length, feedback, and system training. Moreover, the interplay between the different components and additional factors can lead to degraded system performance when adding or manipulating particular components. We explore the interactions and effects of different components and some of the factors with respect to performance. The main contribution of this paper is a better understanding of how to configure filtering systems along with the possible pitfalls of applying conflicting components which harm performance and result in a poor user experience.
Towards realistic known-item topics for the ClueWeb BIBAFull-Text 274-277
  Claudia Hauff; Matthias Hagen; Anna Beyer; Benno Stein
Known-item finding is the task of re-finding and re-accessing an item previously seen. Typical examples of known items include accessed Web sites, received emails, or documents on one's personal desktop. Current research on known-item finding heavily relies on corpora of known-item queries and the respective known items. However, many existing corpora are proprietary and not available to the public (in particular those derived from Web query logs), a fact which does not allow for repeatable research. The existing publicly available corpora either contain automatically generated queries or queries that were manually generated while seeing the known item itself. Hence, we consider these public corpora to be rather artificial in nature.
   In this paper, we propose a methodology to create a known-item topic set that is much more realistic and that is built on top of a large-scale public test corpus. From know-item questions posted on the popular Yahoo! Answers platform we extract queries for known-items in a crowdsourcing setup. Since we ensure that all the known-items correspond to Web pages in the publicly available ClueWeb09 corpus (a large static Web crawl), we provide an environment for repeatable realistic Web-scale known-item searches.
Unobtrusive mobile browsing behaviour tracking tool BIBAFull-Text 278-281
  Matti Lassila; Teemu Pääkkönen; Paavo Arvola; Jaana Kekäläinen; Marko Junkkari
Unobtrusive user tracking is needed in order to achieve realistic test settings for IIR. This demand is stressed, when studying the usage of mobile devices in IR. In this study, we present a model and a tool for observing and recording user's browsing within a retrieved web document. We extend an existing open source program UsaProxy with functionalities for tracking browsing behaviour.
EmSe: initial evaluation of a child-friendly medical search system BIBAFull-Text 282-285
  Carsten Eickhoff; Leif Azzopardi; Djoerd Hiemstra; Franciska de Jong; Arjen de Vries; Doug Dowie; Sergio Duarte; Richard Glassey; Karl Gyllstrom; Frea Kruisinga; Kelly Marshall; Sien Moens; Tamara Polajnar; Frans van der Sluis
When undergoing medical treatment in combination with extended stays in hospitals, children have been frequently found to develop an interest in their condition and the course of treatment. A natural means of searching for related information would be to use a web search engine. The medical domain, however, imposes several key challenges on young and inexperienced searchers, such as difficult terminology, potentially frightening topics or non-objective information offered by lobbyists or pharmaceutical companies. To address these problems, we present the design and usability study of EmSe, a search service for children in a hospital environment.
SCAMP: a tool for conducting interactive information retrieval experiments BIBAFull-Text 286-289
  Gareth Renaud; Leif Azzopardi
Conducting Interactive Information Retrieval (IIR) research is often seen as an arduous and tedious process with a high barrier to entry. This high barrier is due to the overheads in developing and setting up even a simple lab-based IIR experiment. SCAMP (Search ConfigurAtor for experiMenting with PuppyIR) is a web-based tool that we have developed which enables researchers to configure standard IIR experiments. SCAMP provides the infrastructure that handles the major processes within the experimental flow (such as Participant Registration, Consent, Surveys, and the logging and tracking of tasks and participants through the experiment). Consequently, SCAMP reduces the time required to create an experiment and is ideal for undergraduate and masters students who would like to conduct an IIR experiment without extensive development. Furthermore, the tool is extensible. Other features can be easily added to SCAMP to customise the experiments. To evaluate SCAMP, we performed a within-subjects experiment where 48 participants used a web search engine with different search aids (query suggestion, spell correction, etc) to complete various web search tasks -- to increase the difficulty of the search tasks certain query terms were banned. We use this evaluation to showcase the different features of the SCAMP system and report on how participants perform under difficult querying conditions.
Using file system content to organize e-mail BIBAFull-Text 290-293
  Maya Sappelli; Suzan Verberne; Wessel Kraaij
This paper is about using existing directory structures on the file system as models for e-mail classification. This is motivated by the aim to reduce the effort for users to organize their information flow.
   Classifiers were trained on categorized documents and tested on their performance on an unstructured set of e-mail correspondence related to the documents. Even though the documents and e-mails in our corpus belonged to the same categories, the classifiers showed very low accuracy on e-mail classification. More importantly, a learning curve experiment showed that initiating a model with documents can have a negative impact on the overall accuracy that could be achieved on e-mail classification. Features important for e-mail classification are inherently different than those important for document classification.
Correlating perception-oriented aspects in user-centric recommender system evaluation BIBAFull-Text 294-297
  Alan Said; Brijnesh J. Jain; Andreas Lommatzsch; Sahin Albayrak
Research on recommender systems evaluation generally measures the quality of the algorithm, or system, offline, i.e. based on some information retrieval metric, e.g. precision or recall. The metrics do however not always reflect the users' perceptions of the recommendations. Perception-related values are instead often measured through user studies, however the bulk of the work on recommender systems is evaluated through offline analysis. In the work presented in this paper we choose to neglect the quality of the recommender system and instead focus on the similarity of aspects related to users' perception of recommender systems. Based on a user study (N = 132) we show the correlation of concepts such as usefulness, ratings, obviousness, and serendipity from the users' perspectives.
Unlocking radio broadcasts: user needs in sound retrieval BIBAFull-Text 298-301
  Mette Skov; Marianne Lykke
This poster reports the preliminary results of a user study uncovering the information seeking behaviour of humanities scholars dedicated to radio research. The study is part of an interdisciplinary research project on radio culture and auditory resources. The purpose of the study is to inform the design of information architecture and interaction design of a research infrastructure that will enable future radio and audio based research. Results from a questionnaire survey on humanities scholars' research interest and information needs, preferred access points, and indexing levels are reported. Finally, a flexible metadata schema is suggested, that includes both general metadata and highly media and research project specific metadata.
An exploratory study into perceived task complexity, topic specificity and usefulness for integrated search BIBAFull-Text 302-305
  Peter Ingwersen; Christina Lioma; Birger Larsen; Peiling Wang
We investigate the relations between user perceptions of work task complexity, topic specificity, and usefulness of retrieved results. 23 academic researchers submitted detailed descriptions of 65 real-life work tasks in the physics domain, and assessed documents retrieved from an integrated collection consisting of full text research articles in PDF, abstracts, and bibliographic records [6]. Bibliographic records were found to be more precise than full text PDFs, regardless of task complexity and topic specificity. PDFs were found to be more useful. Overall, for higher task complexity and topic specificity bibliographic records demonstrated much higher precision than did PDFs on a four-graded usefulness scale.
SlideDeckFinder: identifying related slide decks based on visual appearance and composition patterns BIBAFull-Text 306-309
  Oliver Brdiczka; Doron Kletter
This paper introduces SlideDeckFinder, a tool integrated into a user's email client enabling the search for similarities between slide decks. The similarity calculations are based on visual correspondence (both from text and images/graphics) as well as slide (re-)composition patterns. The individual slides of different slide decks are first compared by matching their respective visual features extracted from any content such as text and images. The resulting similarity scores between pairs of slides are then the input for calculating the similarity between whole slide decks. Hidden Markov models (HMMs) are used to represent the transformation (in terms of re-arrangements or insertions of new slides) from one slide deck to another, where the state emissions probabilities of the HMM correspond to slide similarity and the transition probabilities represent the likely slide sequence within slide decks. The Viterbi algorithm is finally used to calculate the most likely state sequence (i.e. recomposition pattern) between the slide decks and thus the similarity score. SlideDeckFinder has been evaluated both on its accuracy to compare visual appearance of slides with respect to human perception and its performance to retrieve related slide deck variants.
Evaluation of search quality differences and the impact of personality styles in native and foreign language searching tasks BIBAFull-Text 310-313
  Eszter Józsa; Máté Köles; Anita Komlódi; Károly Hercegfi; Peng Chu
Taking individual differences into consideration is a foundational issue of Human-Computer Interaction research. The current paper examines the differences that arise during native and foreign language information-seeking tasks. Seventeen Hungarian college students, with significant but non-heritage knowledge of English, participated in the study. We examined the impact of search strategies and personality types on search outcome quality. Our results show interesting variations in searchers' success in their native and foreign languages. In-depth search strategies work better and allow searchers to achieve the same success rate in a foreign language as in their native language. More empathy toward the search task also seems to improve results.
Making the news interesting: understanding the relationship between familiarity and interest BIBAFull-Text 314-317
  Frans van der Sluis; Richard J. Glassey; Egon L. van den Broek
News feeds are an important element of information encountering, feeding our (new) interests but also leading to a state of information overload. Current solutions often select information similar to the user's interests. However, long-term interest in one topic, and being highly familiar with that topic, does not necessarily imply an actual interest response will occur when more of the same topic is selected. This study explores how important familiarity is in predicting an interest response. In a study with 30 subjects, interest was manipulated by topical familiarity using novel stimuli from a popular news source. This study shows, within this context, familiarity is moderately important for an interest response: familiarity does indeed make the news interesting, but only to a certain extent. The results set a baseline for predicting interest during information encountering, indicating familiarity is important, but not the only influential variable a system should consider when selecting information for users.
An exploratory study on search behavior in different languages BIBAFull-Text 318-321
  Peng Chu; Eszter Jozsa; Anita Komlodi; Karoly Hercegfi
More and more people search online in a language that is not their native language due to the limited availability of content in underrepresented languages [1]. Most such users search in English as a second language. Very few researchers studied the challenges second language searchers face, even though searching in a foreign language and language proficiency and skills constitute an essential part of the context of searching, which can in return impact the search process and outcome. This exploratory study examines differences in first and second language web searching. Query reformulations from 14 participants who searched in English (first language) and Spanish (second language) and 17 participants who searched in Hungarian (first language) and English (second language) are analyzed and compared. Preliminary results show that searching in a foreign language requires significantly longer time, more query reformulations, and more websites viewed. User feedback also indicates that the search strategy that our participants successfully use in their first language is often much less efficient in their second language.

Doctoral consortium session

Capturing context and mental state of knowledge workers BIBAFull-Text 322
  Saskia Koldijk
We live in an information society, in which effective and efficient interaction with information is crucial. Many people are knowledge workers, whose main job it is to interpret and generate information. Due to their typical working conditions these people often experience stress while working with information [1]. They get overwhelmed by all the available information and often have a fragmented way of working due to inappropriate interruptions, for example by incoming mails with information requests. As a consequence, this way of working can diminish well-being at work, which -- for some people -- may finally result in burn-out.
Personalized support in exploratory search BIBAFull-Text 323
  Daniel T. J. Backhausen
Complex tasks like answering research questions or solving problems require to carry out longitudinal processes where different information objects need to be gathered, collected, interpreted, analyzed, and evaluated [1]. Such a process normally includes several search and exploration sessions where the user interactively digs deeper into a more or less unknown domain.
   This research is driven by the fact that most common systems are designed to fit a general user where users are submitting queries and the retrieval system returns a ranked list of results. Regardless of the user, the query always returns the same list of results. Individual aspects like age, gender, profession or experience are often not taken into account, for example the difference in searching between children and adults. Unfortunately many systems are optimized for lookup searches, expecting that the user is only interested in facts and not in problem solving. Additionally common systems still assume that the user has a static information need which remains unchanged during the seeking process. In each step of the overall seeking process, the user faces a new situation in which knowledge and information need changes. This influences the relevance of information objects and may direct each user individually to different topics, domains, tasks or even search strategies.
   Due to uncertainty and missing knowledge, exploratory search activities need far more assistance like closing the gap between different search sessions, allowing the user to review and continue their search more easily. Moreover the complexity of working tasks and the individual qualifications require personalized support to the searcher. The goal of this research is to investigate a concept assisting the user within such interactive exploratory search activities, allowing an effective information exploration by personalizing the seeking & searching process. Personalized IR systems need to adapt to relevant factors and commit itself to the specific user and the personal search behavior. The user should be guided throughout the searching process, suggesting useful search strategies and effective tactics which matches the users searching behavior and the current situation. To bridge different search sessions, past activities must be visualized in a kind of breadcrumb or timeline. That's why we are currently prototyping a way to visualize the personal Google search history using Timeline JS.
   To further assist the user with strategic search support, it is necessary to be aware of the user herself and specific contextual circumstances which may be relevant to the situation. General information about the user like gender or age but also relevance feedback can be fetched explicitly, allowing the system to adapt in a more coarse grained way (e.g. deciding the way of presenting results). Moreover integrating common used applications (e.g. Evernote) or providing other ways to let the user manage tasks will help to understand the goal of the search activities. For this reason we are currently investigating ways to link search activities with task management.
   Information about the search behavior and indirectly the users knowledge and expertise can be conveyed by logging (e.g. query logs) and examining system interactions. The fetched data should be made transparent to the user, showing what kind of information has been gathered so far. The implicitly gained information has to be refined with the explicit ones and also other contextual data collected tacitly from different interfaces or sensors (e.g. time, location). Bringing it all together will allow a more fine grained way of system adaption and offers new options in assisting the user during the long-term search activities showing personalized search strategies and possible next search steps appropriate to the information need and level of experience.
Nonverbal query driven interactive search systems: a study on language agnostic information access technologies BIBAFull-Text 324
  Viktors Garkavijs; Noriko Kando
This work is concentrated on analyzing possibilities for retrieving and presenting the information available on the web, without the explicit need to formulate precise queries. The research question of our work can be formulated as: "Is it possible to retrieve information without clearly formulated verbal query?"
   For this consortium we present our prototype of interactive image search system called "Gaze Learning Access and Search Engine" (GLASE) version 0.1, which can perform relevance calculation based on gaze data and within-session learning. The search user interface (UI) uses an eye-tracker as an input device and employs a relevance re-ranking algorithm based on the gaze length. The preliminary experimental results showed that using our gaze-driven system reduced the task completion time an average of 13.7% in a search session.
Fiction retrieval in enriched library systems BIBAFull-Text 325
  Anna Mikkonen
Recreational reading is an important part of people's everyday life in many countries. Borrowing books and leisure reading are the most popular activities among library users, and most of the books borrowed in the public libraries are fiction. Although it is known that leisure reading concentrates mostly on fiction, and fiction readers make up a large portion of the public library clientele, we have little information on how readers are currently using library catalogues for fiction searching. Current knowledge on users' search behavior with digital systems has mainly focused on retrieving for non-fiction. Also in system development the focus has been on retrieving for non-fiction. In spite of the development of library catalogues and digital content description methods for fiction collections, research on the usability of new content description tools as a part of fiction retrieval in online catalogues is lacking. There is a gap in our understanding of how people currently search for recreational reading material in online catalogues and how these enriched catalogues support this activity in different reader groups.
Collaborative information retrieval in tourism: a study of user search behaviour, user interface and information retrieval performance BIBAFull-Text 326
  Abu Shamim Mohammad Arif; Jia Tina Du; Ivan Lee
Tourism information is dynamic and travel routes and decisions are dependent on highly varying factors such as perceived attractive sites, weather conditions, prices, transportation, accommodation, holidays, economic changes and so on. Travel guidebooks and present search engines completely lack this potential for dialog. Travel information search activities involved planning, decision making, anticipation of the trip with other people. The community (network of people) may act as a gateway to the information repository, when a tourist is not able to find the right information himself/herself or does not know about his/her information need. This leads to the collaboration for searching tourism information. Considering the following issues we are motivated to study this topic.
Supervision of learning methods in user data interpretation BIBAFull-Text 327
  Maya Sappelli; Suzan Verberne; Wessel Kraaij
Knowledge workers need support in their daily activities to handle the information stream they encounter continuously (e.g. e-mail, search results etc.). One method for this is to place the information in context, i.e. to what activity is the information related? For this purpose, contexts need to be recognized. Additionally, this should happen in an unobtrusive way such that the worker is not disrupted and can remain focused at his work-related activities.
How far will you go?: using need for closure and information scent to model search stopping behavior BIBAFull-Text 328
  Wan-Ching Wu
Online searching plays a central role in daily life. During online searching, people first decide which queries to submit to an information retrieval system, after that they decide which links to click on, and based on the quality of search result content, they further determine whether to stop viewing the current returned result set or to refine the query. At some point a person decides to terminate searching. For tasks that require a single answer such as a navigation task, it is relatively clear when a person decides to stop searching -- he or she has either found, or not found, an answer to his or her question. However, for tasks that have no obvious end-point, it is unclear when and why people decide to stop searching and more importantly, it is unclear how to model such behavior. The research question of deciding when information is enough to complete one's task is mostly investigated under the term "search stopping behavior" [3], even though no formal definition has been given in the literature.