Humberto Cavallin, W. Mike Martin, Ann Heylighen
In this paper we describe our findings regarding the role of context in usability evaluation, particularly how the nature of the tasks can affect the users’ perception of the performance of a particular application. Our findings show a relationship between the variation in the nature of the tasks used for usability evaluation on the one hand, and the way in which subjects evaluated these applications afterwards by using user-administered questionnaires on the other hand. These findings contradict the absolute benchmarking goal of some of these tools, thus raising questions about the possibility of achieving that kind of benchmarks in software usability evaluation, and about how comparative measurements of the benefits of software and technology take place in laboratory conditions.
Cavallin, H., Martin, W. M., & Heylighen, A. (2005b). Mind-ing the Task. The role of context in usability research. SID2005.