An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices

There is a clear need for evaluation methods that are specifically suited to mobile device evaluation, largely due to the vast differences between traditional desktop computing and mobile computing. One difference of particular interest that needs to be accounted for is that mobile computing devices are frequently used while the user is in motion, in contrast to desktop computing. This study aims to validate the appropriateness of two evaluation methods that vary in representativeness of mobility, one that uses a treadmill to simulate motion and another that uses a controlled walking scenario. The results lead to preliminary guidelines based on study objectives for researchers wishing to use more appropriate evaluation methodologies for empirical, data-driven mobile computing studies. The guidelines indicate that using a treadmill for mobile evaluation can yield representative performance measures, whereas a controlled walking scenario is more likely to adequately simulate the actual user experience.