Early childhood preservice teachers’ debugging block-based programs: An eye tracking study
Learning computational skills such as programming and debugging is very important for K-12 students given the increasing need of workforce proficient in computing technologies. Programming is an intricate cognitive task that entails iteratively creating and revising programs to create an artifact. Central to programming is debugging, which consists of systematically identifying and fixing program errors. Given its central role, debugging should be explicitly taught to early childhood preservice teachers so they can support their future students’ learning to program and debug errors. In this study, we propose using eye-tracking data and cued retrospective reporting to assess preservice teachers’ cognitive strategies while debugging. Several eye-tracking studies have investigated learners’ debugging strategies though the literature lacks studies (a) conducted with early childhood preservice teachers and (b) that focus on block-based programming languages, such as Scratch. The present study addresses this gap in the literature. This study used mixed methods to triangulate quantitative findings from eye movement analysis and qualitative findings about employed debugging strategies into the creation of descriptive themes. Results showed that participants developed strategies such as simultaneous review of output and code, use of beacons to narrow down the area to be debugged, and eye fixation on output to form hypotheses. But most often, debugging was not informed by a hypothesis, which led to trial and error. Study limitations and directions for future research are discussed.
Alqadi, B. S., & Maletic, J. I. (2017). An empirical study of debugging patterns among novice programmers. In S. Edwards, M. Caspersen, T. Barnes, & D. Garcia (Eds.), Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education—SIGCSE’17 (pp. 15–20). New York, NY: ACM Press. https://doi.org/10.1145/3017680.3017761
Aschwanden, C., & Crosby, M. (2006). Code scanning patterns in program comprehension. Paper presented at the 39th Hawaii International Conference on System Sciences. Retrieved from https://pdfs.semanticscholar.org/1153/854b79709f4d552959eebf9f7b2c0431f63a.pdf
Basu, S. (2016). Fostering synergistic learning of computational thinking and middle school science in computer-based intelligent learning environments (Unpublished doctoral dissertation). Vanderbilt University, Nashville, TN, USA.
Bau, D. (2015). Droplet, a block-based editor for text code. Journal of Computing Sciences in Colleges, 30(6), 138–144. Retrieved from https://dl.acm.org/doi/10.5555/2753024.2753052
Bednarik, R. (2012). Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies, 70(2), 143-155. https://doi.org/10.1016/j.ijhcs.2011.09.003
Bednarik, R., Schulte, C., Budde, L., Heinemann, B., & Vrzakova, H. (2018). Eye-movement modeling examples in source code comprehension: A classroom study. In M. Joy, & P. Ihantola (Eds.), Proceedings of the 18th Koli Calling International Conference on Computing Education Research (pp. 1-8). New York, NY: ACM Press. https://doi.org/10.1145/3279720.3279722
Beller, M., Spruit, N., Spinellis, D., & Zaidman, A. (2018). On the dichotomy of debugging behavior among programmers. In P. Lago, & M. Young (Eds.), Proceedings of the 40th International Conference on Software Engineering - ICSE ’18 (pp. 1–12). New York, NY: ACM Press. https://doi.org/10.1145/3180155.3180175
Braught, G., Wahls, T., & Marlin Eby, L. (2011). The case for pair programming in the computer science classroom. ACM Transactions on Computing Education, 11(1), 1–21. https://doi.org/10.1145/1921607.1921609
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. Paper presented at the 2012 Annual Meeting of the American Educational Research Association. Retrieved from http://scratched.gse.harvard.edu/ct/files/AERA2012.pdf
Brooks, R. E. (1983). Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies, 18(6), 543-554. https://doi.org/10.1016/S0020-7373(83)80031-5
Burke, Q. (2012). The markings of a new pencil: Introducing programming-as-writing in the middle school classroom. Journal of Media Literacy Education, 4(2), 121–135.
Busjahn, T., Schulte, C., & Busjahn, A. (2011). Analysis of code reading to gain more insight in program comprehension. In A. Korhonen, & R. McCartney (Eds.), Proceedings of the 11th ACM Koli Calling International Conference on Computing Education Research (pp. 1-9). New York, NY: ACM Press. https://doi.org/10.1145/2094131.2094133
Code.org. (n.d.). CS fundamentals unplugged. Retrieved from https://code.org/curriculum/unplugged
Computer Science Teachers Association & International Society for Technology in Education (2011). Computational thinking for all (2nd ed.). Retrieved from https://www.iste.org/explore/articleDetail?articleid=152&category=Solutions&article=Computational-thinking-for-all
Cooper, S., Dann, W., & Pausch, R. (2000). Developing algorithmic thinking with Alice. In D. Colton (Ed.), The Proceedings of the Information Systems Education Conference, (pp. 506–539). Philadelphia, PA: Education Special Interest Group of the Association of Information Technology Professionals and the Foundation for Information Technology Education. Retrieved from https://pdfs.semanticscholar.org/15c0/016285c067f2f8dbbf2d7d59f9ffceb9237f.pdf
Crosby, M., Scholtz, J., & Wiedenbeck, S. (2002). The roles beacons play in comprehension for novice and expert programmers. In J. Kuljis, L. Baldwin, & R. Scoble (Eds.), Proceedings of the 14th Annual Workshop on Psychology of Programming Interest Group (pp. 58-73). London, UK: Brunel University.
Fitzgerald, S., Lewandowski, G., McCauley, R., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: Finding, fixing, and flailing, a multi-institutional study of novice debuggers. Computer Science Education, 18(2), 93-116. https://doi.org/10.1080/08993400802114508
Ge, X., & Land, S. M. (2004). A conceptual framework for scaffolding III-structured problem-solving processes using question prompts and peer interactions. Educational Technology Research and Development, 52(2), 5-22. https://doi.org/10.1007/bf02504836
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine. https://doi.org/10.1097/00006199-196807000-00014
Google (n.d.). Computational thinking for educators [online course]. Retrieved from https://computationalthinkingcourse.withgoogle.com/course?use_last_location=true
Google, & Gallup. (2015). Searching for computer science: Access and barriers in U.S. K-12 education. Retrieved from http://g.co/cseduresearch
Gould, J., & Drongowski, P. (1974). An exploratory study of computer program debugging. Human Factors, 16(3), 258-277. https://doi.org/10.1177%2F001872087401600308
Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
Greene, J. C. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2(1), 7-22. https://doi.org/10.1177/1558689807309969
Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051
Hejmady, P., & Narayanan, N. H. (2012). Visual attention patterns during program debugging with an IDE. In S. N. Spencer (Ed.), Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’12) (pp. 197-200). New York, NY: ACM Press. https://doi.org/10.1145/2168556.2168592
Isong, B. (2014). A methodology for teaching computer programming: First year students’ perspective. International Journal of Modern Education and Computer Science, 6(9), 15–21. https://doi.org/10.5815/ijmecs.2014.09.03
Jadud, M. C. (2005). A first look at novice compilation behaviour using BlueJ. Computer Science Education, 15(1), 25-40. https://doi.org/10.1080/08993400500056530
Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4(1), 39-103. https://doi.org/10.1207/s15327809jls0401_2
K-12 Computer Science Framework Steering Committee (2016). K–12 computer science framework. Retrieved from http://www.k12cs.org
Kalelioğlu, F. (2015). A new way of teaching programming skills to K-12 students: Code.org. Computers in Human Behavior, 52(1), 200–210. https://doi.org/10.1016/j.chb.2015.05.047
Kazimoglu, C., Kiernan, M., Bacon, L., & MacKinnon, L. (2012). Learning programming at the computational thinking level via digital game-play. Procedia Computer Science, 9(1), 522–531. https://doi.org/10.1016/j.procs.2012.04.056
Kim, C., Yuan, J., Oh, J., Shin, M., & Hill, R. B. (2016). Productive struggle during inquiry learning. Paper presented at the European Association for Research on Learning & Instruction SIG 20 & SIG 26 Meetings, Ghent, Belgium.
Kim, C., Yuan, J., Vasconcelos, L., Shin, M., & Hill, R. B. (2018). Debugging during block-based programming. Instructional Science, 46(5), 767–787. https://doi.org/10.1007/s11251-018-9453-5
Leech, N. L., & Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs. Quality & Quantity, 43(2), 265–275. https://doi.org/10.1007/s11135-007-9105-3
Leedy, P. D., & Ormrod, J. E. (2013). Practical research: Planning and design. Boston: Pearson.
Leonard, J., Buss, A., Gamboa, R., Mitchell, M., Fashola, O. S., Hubert, T., & Almughyirah, S. (2016). Using robotics and game design to enhance children’s self-efficacy, STEM attitudes, and computational thinking skills. Journal of Science Education and Technology, 25(6), 860–876. https://doi.org/10.1007/s10956-016-9628-2
Lin, Y. T., Wu, C. C., Hou, T. Y., Lin, Y. C., Yang, F. Y., & Chang, C. H. (2015). Tracking students’ cognitive processes during program debugging — An eye-movement approach. IEEE Transactions on Education, 59(3), 175-186. https://doi.org/10.1109/TE.2015.2487341
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41(1), 51–61. https://doi.org/10.1016/j.chb.2014.09.012
McCauley, R., Fitzgerald, S., Lewandowski, G., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358– 368. https://doi.org/10.1037/0022-0618.104.22.1688
National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: National Academies Press.
Obaidellah, U., Al Haek, M., & Cheng, P. C. H. (2018). A survey on the usage of eye-tracking in computer programming. ACM Computing Surveys, 51(1), 1–58. https://doi.org/10.1145/3145904
Obama, B. (2016). Computer science for all. Retrieved from https://obamawhitehouse.archives.gov/blog/2016/01/30/computer-science-all
Papavlasopoulou, S., Sharma, K., Giannakos, M., & Jaccheri, L. (2017). Using eye-tracking to unveil differences between kids and teens in coding activities. In P. Blikstein, & D. Abrahamson (Eds.), Proceedings of the 2017 Conference on Interaction Design and Children (pp. 171-181). New York, NY: ACM Press. https://doi.org/10.1145/3078072.3079740
Paul, A. M. (2016). The coding revolution. Scientific American, 315(2), 42–49. https://doi.org/10.1038/scientificamerican0816-42
Pietinen, S., Bednarik, R., Glotova, T., Tenhunen, V., & Tukiainen, M. (2008). A method to study attention aspects of collaboration: Eye-tracking pair programmers simultaneously. In K. Räihä, & A. T. Duchowski (Eds.), Proceedings of the 2008 Symposium on Eye Tracking Research and Applications (pp. 39-42). Savannah, GA: ACM Press. https://doi.org/10.1145/1344471.1344480
Proctor, C. (2019). Measuring the computational in computational participation: Debugging interactive stories in middle school computer science. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings (Vol. 1, pp. 104–111). Lyon, France: International Society of the Learning Sciences.
Project Growing Up Thinking Scientifically [GUTS] (n.d.). Retrieved from http://www.projectguts.org/resources
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., … Kafai, Y. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67. https://doi.org/10.1145/1592761.1592779
Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, CA: SAGE.
Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351–380. https://doi.org/10.1007/s10639-012-9240-x
Sharafi, Z., Soh, Z., & Guéhéneuc, Y. (2015). A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology, 67(1), 79–107. http://dx.doi.org/10.1016/j.infsof.2015.06.008
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22(1), 142–158. https://doi.org/10.1016/j.edurev.2017.09.003
Simon, B., Bouvier, D., Chen, T., Lewandowski, G., McCartney, R., & Sanders, K. (2008). Common sense computing (episode 4): Debugging. Computer Science Education, 18(2), 117-133. https://doi.org/10.1080/08993400802114698
Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage.
Sun, J. C., & Hsu, K. Y. (2019). A smart eye-tracking feedback scaffolding approach to improving students’ learning self-efficacy and performance in a C programming course. Computers in Human Behavior, 95(1), 66–72. https://doi.org/10.1016/j.chb.2019.01.036
van Gog, T., Paas, F., van Merriënboer, J. J., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied, 11(4), 237-244. https://doi.org/10.1037/1076-898X.11.4.237
Vasconcelos, L., & Kim, C. (2019). Coding in scientific modeling lessons (CS-ModeL). Educational Technology Research and Development. https://doi.org/10.1007/s11423-019-09724-w
Vihavainen, A., Airaksinen, J., & Watson, C. (2014). A systematic review of approaches for teaching introductory programming and their influence on success. In Proceedings of the Tenth Annual Conference on International Computing Education Research - ICER ’14, (pp. 19–26). New York, NY: ACM Press. https://doi.org/10.1145/2632320.2632349
Yen, C., Wu, P., & Lin, C. (2012). Analysis of experts’ and novices’ thinking process in program debugging. In K. C. Li, F. L. Wang, K. S. Yuen, S. K. S. Cheung, & R. Kwan (Eds.), Engaging Learners Through Emerging Technologies (Vol. 302, pp. 122–134). https://doi.org/10.1007/978-3-642-31398-1_12
Yukselturk, E., & Altiok, S. (2017). An investigation of the effects of programming with Scratch on the preservice IT teachers’ self-efficacy perceptions and attitudes towards computer programming. British Journal of Educational Technology, 48(3), 789–801. https://doi.org/10.1111/bjet.12453
Yusuf, S., Kagdi, H. H., & Maletic, J. I. (2007). Assessing the comprehension of UML class diagrams via eye tracking. In Proceedings of 15th IEEE International Conference on Program Comprehension (pp. 113–122). Alberta, Canada: IEEE. https://doi.org/10.1109/ICPC.2007.10
Copyright (c) 2020 Journal of Childhood, Education & Society
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Author(s) accept that the Owner of Journal of Childhood, Education & Society, the Editor, Associate Editors, Reviewers and the Editorial Board cannot be hold responsible regarding the scope, the findings, the discussion and conclusion of the manuscript submitted.
Author(s) declare to the editorship of Journal of Childhood, Education & Society that the manuscript is original and has not been published anywhere else or is not under evaluation process for any other journal.
Author(s) approve that they grant Journal of Childhood, Education & Society as the sole and exclusive right and license to publish for the full legal term of copyright of their manuscript concurring with article 5846 / 22-23-25 while they retain copyright in the work.