An investigation of outlying performance in menu search

It has been noticed that in usability tests there is always one or more participants who take a long time to complete the task compared with other participants. Such participants appear as outliers in the collected time-on-task data. While such outliers can skew any statistical analysis, at the same time they may represent genuine problems with the interaction. This research started with an exploratory study that investigated outlying performance in usability testing practices, to find out how outliers are interpreted and treated by practitioners. The practitioners interviewed in this study seem aware of the regular occurrence of outliers. They tend to link outlying performance cases to individual differences instead of linking them to usability problems. However, there appears to be no systematic approach to addressing them. This research focuses on investigating outlying performance in menu search. Previous work suggests that the perceived menu semantics plays a role in outlying menu search performance. In this context, menu semantics refers both to the names of the menu items and to the names of the menu titles. Additionally, it refers to the semantic organization of the menu items. The series of studies conducted in this research used different methodologies to check whether the perceived menu semantics plays a role in outlying menu search performance. The results suggest that perceived menu semantics may play a role in outlying menu search performance. Moreover, this research checked whether outlying menu search performance is due to specific individuals. The results suggest that outlying menu search performance can be due to individual differences. The practical implication of this research is that outliers are a fact that should not be ignored. They should be considered and the reasons behind them should be identified. Consequently, based on the identified reasons, there is a possibility of improving the design for outliers.