Text size
  • Small
  • Medium
  • Large
  • Standard
  • Blue text on blue
  • High contrast (Yellow text on black)
  • Blue text on beige
Your location:

Remote user testing shows high task incompletion rate

Source: UN, 22 September 2001

Microsoft's comparison studies between lab-based and web-based user testing reveal benefits in adopting both methods, but high task incompletion rates remotely.

Using comparison studies between lab-based product tests and web-based equivalents, Microsoft usability engineers have begun to evaluate the differences between the two techniques in terms of results and effectiveness.

In preliminary tests using 12 lab-based users and 25 web-based users undertaking 20 and 6 tasks respectively, Richard Jacques and Hernan Savastano found that they could collect information on success rates, pathways taken, duration of tasks, and users' thoughts through questionnaires using either method.

But, while remote testing has the immediate appeal of low lab involvement, the speedy collection of large samples and the opportunity of soliciting information widely, testers found that task completion rates suffered, with 20% of specified tasks abandoned before completion.

These initial tests were not conducted using the same controllable variables. For instance, web testers were given 45 minutes to complete their task, whereas those testing using the same questions in the lab were given as long as they required and this might well have distorted the results.

But one insurmountable weakness of using a remote method, commented Jacques, is that you cannot observe and do not hear the user during the testing process.

The team intend to conduct further, more tightly controlled, comparison studies in the months coming to establish the scope of each method's usefulness.