An investigation of outlying performance in menu search

It has been noticed that in usability tests there is always one or more participants who
take a long time to complete the task compared with other participants. Such
participants appear as outliers in the collected time-on-task data. While such outliers
can skew any statistical analysis, at the same time they may represent genuine
problems with the interaction.
This research started with an exploratory study that investigated outlying performance
in usability testing practices, to find out how outliers are interpreted and treated by
practitioners. The practitioners interviewed in this study seem aware of the regular
occurrence of outliers. They tend to link outlying performance cases to individual
differences instead of linking them to usability problems. However, there appears to
be no systematic approach to addressing them.
This research focuses on investigating outlying performance in menu search. Previous
work suggests that the perceived menu semantics plays a role in outlying menu search
performance. In this context, menu semantics refers both to the names of the menu
items and to the names of the menu titles. Additionally, it refers to the semantic
organization of the menu items. The series of studies conducted in this research used
different methodologies to check whether the perceived menu semantics plays a role
in outlying menu search performance. The results suggest that perceived menu
semantics may play a role in outlying menu search performance.
Moreover, this research checked whether outlying menu search performance is due to
specific individuals. The results suggest that outlying menu search performance can
be due to individual differences.
The practical implication of this research is that outliers are a fact that should not be
ignored. They should be considered and the reasons behind them should be identified.
Consequently, based on the identified reasons, there is a possibility of improving the
design for outliers.