ADAPTIVE DECISION MAKING IN MENU SEARCH THE ROLE OF INTERDEPENDENCE AND PAST EXPERIENCE ON

INTERDEPENDENCE AND PAST EXPERIENCE ON LINK SELECTION

ABSTRACT

Duncan P. Brumby Andrew Howes School of Psychology Cardiff University Cardiff, CF10 3YG BrumbyDP@Cardiff.ac.uk

When searching a web page for information that is relevant to some search goal, people focus on labelled links and estimate the likelihood that these links will lead to the goal. Evidence presented in this paper supports the idea that people adapt their search strategy to the statistics of the environment and that assessments of menu items are interdependent. The work contributes to the development of a cognitive model of how people search web sites, which may serve as an engineering tool to support web design.

Keywords

Visual search, menu search, cognitive modeling, web design, World Wide Web, usability, eye-tracking

expected information gain of making another assessment exceeds the cost of the assessment. In this rational model there is interdependency between the assessments of each item, which occurs due to a normalization assumption (see [8] for details). Young’s model is interesting because it is able to account for the observation that users do not often assess all of the items in the choice set prior to selection [7]. A novel and empirically untested prediction to emerge from the normalization assumption is that the relevance of the distracter items to the search goal will also affect the decision to select an item. More specifically, the normalization assumption predicts that the presence of lower relevance distracters will result in fewer items being assessed.

Another potential influence on the decision to select an item or assess further items is the history of previous experience. In problem solving tasks an operator is more likely to be selected if it was previously successful [4]. In menu selection it is an open question as to whether previous history of evaluation leads to more or less evaluation.

The aim of our initial study was to demonstrate that the decision to select an item is not simply determined by the relevance of the goal item to the search goal, but is strongly influenced by the relevance of the remaining distracter items and also by past experience. In particular, we predicted that people would assess fewer distracters when the relevance of the distracters was lower.

1. INTRODUCTION

Modern computer interfaces typically require the user to interact with a menu system. For example, consider searching a newly encountered web page for information that is relevant to the achievement of some search goal. The user at first does not know which menu item to select in pursuit of the goal information. Menu items must therefore be assessed to provide an estimate of the subjective relevance to the search goal. Estimating the relevance of an item to the information goal serves as a heuristic to prune the potentially vast search space that may exist in a World Wide Web (WWW) search [6]. The current research program is concerned with the strategy people use to search a newly encountered web page in pursuit of a goal. More specifically, we are concerned with the choice facing the user during menu search: that of selecting the current item or continuing to assess items. An obvious influence on the decision to select an item is the relevance of the goal item to the task description [2]. Indeed, previous cognitive models of menu search assume that the relevance of all of the items displayed in a menu are assessed and that the highest rank ordered item [6], or the first item that exceeds a selection threshold [5] is selected.

Interestingly, Young [8] has proposed a rational model of menu search in which items are assessed so long as the

2. METHOD

The menu search tasks used were derived from responses to a web usage survey posted to undergraduate students at Cardiff University. Menu items for each search task were sampled from WWW sites provided by respondents. The quality of the items was rated to assess the degree to which an item was relevant to the achievement of the search goal. A 2x2 mixed design was used with the difficulty of previous trials as a between-subjects factor and quality of distracter items as a within-subjects factor. Trials were split between filler and critical trials, with most of the filler trials occurring in the initial half of the experiment. Filler trials were used to manipulate the success rate of participants first selection. Participants (n=36) were native

English speakers. Eye-tracking was performed using an ASL Pan/Tilt optic eye-tracking system. The focus of the study was on eye-tracking data for the critical trials, which all participants completed. On the critical trials the distracter items were either moderate or very bad in quality, with the quality of the goal item being approximately equal across all critical trials.

3. RESULTS

We analysed the number of items that were fixated on critical trials in which the participant accurately selected the goal item on the first selection. It was found that participants on average assessed half of the items in the menu (M = 7.84, SD = 2.46) and the number of items that were assessed was affected by the quality of the distracter items (Figure 1). A 2*2 ANOVA revealed a significant main effect for the quality of distracter items (F (1, 34) = 24.27, p<.001) and a significant trend for the difficulty of the previous trials (F (1, 34) = 2.96, p<.1). There was no interaction. We also analysed the number of fixations that occurred after the goal was fixated, revealing a significant main effect for both the quality of distracter items (F (1, 34) = 25.48, p<.001) and difficulty of the previous trials (F (1, 34) = 4.27, p<.05). There was no interaction. 12

Moderate Distracters 10Very Bad Distracters 8 6 4

2 0 HardEasyFigure 1 The effect Difficulty of Previous Trialsof quality of distracter items and difficulty of previous trials on the number of items fixated (error bars +/- 1 standard error)

4. DISCUSSION AND FUTURE DIRECTIONS

The findings of our initial study suggest that the choice between continued assessment and selection in menu search is not entirely determined by the relevance of the selected item to the search goal, but is also determined by the relevance of the assessed distracter items. When people select items from a menu or a web page they do not assess all of the items; moreover, they assess a smaller subset of the items available as the distracter items that they do assess get less attractive. This finding provides empirical support for the idea that assessments of menu items are interdependent. The choice between evaluation and selection was also found to be affected by past experience, a finding that cannot be accounted for by previous models of menu search.

Our initial modeling work has extended Young’s [8] rational model of menu search. The current model provides

a good fit to the data, accounting for the effect of the quality of the distracter items on the number of items fixated prior to the first selection. Future modeling work may be extended in the ACT-R framework [1] as the production rule learning mechanism in the architecture is well suited to capturing the influence of past experience on operator selection over consecutive trials [4].

In future studies we hope to extend our research on single-page menu search, to search in multi-page web sites. Such an extension will offer the potential to delineate factors that influence people’s navigation decisions through web sites, i.e. between selecting a forward link or choosing to backup to a previously visited page. Previous research in this area, using toy web sites in which participants searched binary choice menu pages, found that the decision to backup was governed by memory for the quality of the unselected item on the previous menu page [3]. Future work will extend this research by using ecologically valid materials and increasing the choice set size of the menus. The overarching motivation for this research is to develop cognitive models of how people search web sites and provide clear and unambiguous information to support future web design.

5. REFERENCES

[1] Anderson, J.R., & Lebiere, C. (1998). The atomic

components of thought. Mahwah, NJ: Erlbaum [2] Franzke, M. (1995). Turning research into practice:

characteristics of display-based interaction.

Conference on Human Factors in Computing Systems, CHI’95. [3] Howes, A., Payne, S.J., & Richardson, J. (2002). An

instance-based model of the effect of previous choices on the control of interactive search. Proceedings of the 24th Annual Meeting of the Cognitive Science Society. [4] Lovett, M.C., & Anderson, J.R. (1996). History of

success and current context in problem solving:

combined influences of operator selection. Cognitive Psychology, 31, 168-217 [5] Miller, C.S., & Remington, R.W. (2001). Modeling an

opportunistic strategy for information navigation. Proceedings of the 23rd Annual Conference of the Cognitive Science Society. [6] Pirolli, P., & Card, S. (1999). Information Foraging.

Psychological Review, 106, 3-675 [7] Rieman, J. (1994). Learning strategies and

exploratory behaviour of interactive computer users. PhD dissertation, Dep. Comp. Science, University of Colorado, Boulder, CL. [8] Young, R.M. (1998) Rational Analysis of exploratory

choice. In M. Oaksford & N. Chater (Eds.). Rational Models of Cognition. Oxford: Oxford University Press.

Number of Items Fixated