Towards automating fixation correction for source code

ABSTRACT

During eye-tracking studies there is a possibility for the actual fixation to shift a little when recorded. The cause of this shift could be due to various reasons such as the accuracy of the calibration or drift. Researchers usually correct fixations manually. Manual corrections are error prone especially if done on large samples for extended periods. There is also no guarantee that two corrections done by different people on the same data set will be consistent with each other. In order to solve this problem, we introduce an attempt at automatically correcting fixations that uses a variable offset for groups of fixations. Our focus is on source code, which is read differently than natural language requiring an algorithm that adapts to these differences. We introduce a Hill Climbing algorithm that shifts fixations to a best-fit location based on a scoring function. In order to evaluate the algorithm's effectiveness, we compare the automatically corrected fixations against a set of manually corrected ones, giving us an accuracy of 89%. These findings are discussed with additional ways to improve the algorithm.

References

1. Busjahn, T., Bednarik, R., Begel, A., Crosby, M., Paterson, J. H., Schulte, C., Sharif, B., and Tamm, S. 2015. Eye movements in code reading: Relaxing the linear order. In Intl. Conference on Program Comprehension 2015, 255--265.
2. Carl, M. 2013. Dynamic programming for re-mapping noisy fixations in translation tasks. Journal of Eye Movement Research 6, 2, 1--11.
3. Cerrolaza, J. J., Villanueva, A., Villanueva, M., and Cabeza, R. 2012. Error characterization and compensation in eye tracking systems. In ETRA, 205--208.
4. Cohen, A. L. 2012. Software for the automatic correction of recorded eye fixation locations in reading experiments. Behavior research methods, 679--683.
5. Drewes, J., Masson, G. S., and Montagnini, A. 2012. Shifts in reported gaze position due to changes in pupil size: ground truth and compensation. In ETRA, ACM, Santa Barbara, California, 209--212.
6. Hornof, A. J., and Halverson, T. 2002. Cleaning up systematic error in eye-tracking data by using required fixation locations. Behavior Research Methods, Instruments, & Computers 34, 4 (Nov.), 592--604.
7. Hyrskykari, A. 2006. Utilizing eye movements: Overcoming inaccuracy while tracking the focus of attention during reading. Attention aware systems Special issue: Attention aware systems 22, 4 (July), 657--671.
8. Lohmeier, S. 2015. Experimental Evaluation and Modelling of the Comprehension of Indirect Anaphors in a Programming Language. Master's thesis, Technische Universität Berlin, Germany.
9. Mishra, A., Carl, M., and Bhattacharya, P. 2012. A heuristic-based approach for systematic error correction of gaze data for reading. In Workshop on Eye-tracking and Natural Language Processing, COLING 2012, 71--80.
10. Nüssli, M.-A. 2011. Dual Eye-Tracking Methods for the Study of Remote Collaborative Problem Solving. PhD thesis, École Polytechnique Fédérale de Lausanne, Switzerland.
11. Nyström, M., Andersson, R., Holmqvist, K., and van de Weijer, J. 2012. The influence of calibration method and eye physiology on eyetracking data quality. Behavior Research Methods 45, 1, 272--288.
12. Sharafi, Z., Soh, Z., and Guéhéneuc, Y.-G. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Inf. Softw. Technol. 67, C (Nov.), 79--107.
13. Wyatt, H. J. 2010. The human pupil and the use of video-based eyetrackers. Vision Research 50, 19 (Sept.), 1982--1988.
14. Zhang, Y., and Hornof, A. J. 2014. Easy post-hoc spatial recalibration of eye tracking data. In Proceedings of the Symposium on Eye Tracking Research and Applications, ACM, Safety Harbor, Florida, 95--98.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • Heuristic evaluations

  • Visualization

    • Visualization application domains

      • Visual analytics

• Theory of computation

  • Design and analysis of algorithms