Συστήματα αξιολόγησης της Υπολογιστικής Σκέψης στην εκπαίδευση: Βιβλιογραφική επισκόπηση

PDF
Δημοσιευμένα: ene 1, 2019
DOI: https://doi.org/10.12681/thete.39993
Λέξεις-κλειδιά:
υπολογιστική σκέψη αξιολόγηση προγραμματισμός
Εμμανουήλ Πουλάκης
Παιδαγωγικό Τμήμα Δημοτικής Εκπαίδευσης, Πανεπιστήμιο Θεσσαλίας
Παναγιώτης Πολίτης
Παιδαγωγικό Τμήμα Δημοτικής Εκπαίδευσης, Πανεπιστήμιο Θεσσαλίας
Περίληψη

Το ενδιαφέρον για την Υπολογιστική Σκέψη (ΥΣ) αυξάνεται παγκοσμίως τα τελευταία χρόνια. Παρότι δημιουργείται υλικό για την ανάπτυξή της και προτείνονται προσεγγίσεις για την αξιολόγησή της, δεν υπάρχει ακόμα κοινά αποδεκτός ορισμός. Η αξιολόγηση της ΥΣ αποτελεί απαραίτητη προϋπόθεση για την επιτυχή εισαγωγή της σε προγράμματα σπουδών. Το άρθρο αυτό αποτελεί μια επισκόπηση των ερευνών που έχουν γίνει τα τελευταία χρόνια για την αξιολόγηση της ΥΣ. Οι έρευνες που μελετήθηκαν, ανάλογα με την προσέγγιση που χρησιμοποιούν, κατηγοριοποιήθηκαν σε τρεις ομάδες: α) αξιολόγηση της ΥΣ σε συγκεκριμένα προγραμματιστικά περιβάλλοντα, β) με κριτήρια αξιολόγησης ΥΣ και/ή ψυχομετρικά εργαλεία, και γ) με πολλαπλές μεθόδους αξιολόγησης της ΥΣ. Τα κυριότερα προβλήματα στην αξιολόγηση της ΥΣ είναι η αδυναμία κάλυψης όλων των εννοιών κι ηλικιών των μαθητών και η σχετική ανεπάρκεια των εργαλείων να αξιολογήσουν αυτόνομα κι επαρκώς την ΥΣ. Από τη βιβλιογραφία προκύπτουν προτάσεις για αξιολόγηση της ΥΣ, ενώ η χρήση πολλαπλών μεθόδων αξιολόγησης συνεχίζει να είναι η πλέον ενδεδειγμένη και έγκυρη προσέγγιση.

Λεπτομέρειες άρθρου
  • Ενότητα
  • Articles
Λήψεις
Τα δεδομένα λήψης δεν είναι ακόμη διαθέσιμα.
Αναφορές
Aho, A. V. (2012). Computation and computational thinking. Computer Journal, 55, 832–835.
Allsop, Y. (2018). Assessing computational thinking process using a multiple evaluation approach. International Journal of Child-Computer Interaction, 19, 30–55.
Ambrósio, A.P., Georges, F., & Xavier, C. (2014). Digital ink for cognitive assessment of computational thinking. In M. Castro & E. Tovar (eds.), 2014 IEEE Frontiers in Education Conference (pp. 1520-1526). New Jersey: IEEE.
Araujo, A.L.S.O., Andrade, W.L., & Guerrero, D.D.S. (2016). A systematic mapping study on assessing computational thinking abilities. In S. Frezza, D. Onipede, K. Vernaza & M. Ford (eds.), 2016 IEEE Frontiers in Education Conference (pp. 1-9). New Jersey: IEEE.
Araujo, A.L.S.O., Andrade, W.L., Guerrero, D.D.S., & Melo, M.R.A. (2019). How many abilities can we measure in Computational Thinking?: A study on Bebras Challenge. In E.K. Hawthorne, M.A. Pérez-Quiñones, S. Heckman & J. Zhang (eds.), Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 545-551). New York: ACM.
Araujo, A.L.S.O., Santos, J.S., Andrade, W.L., Guerrero, D.D.S., & Dagienė, V. (2017). Exploring computational thinking assessment in introductory programming courses. In W. Oakes (ed.), 2016 IEEE Frontiers in Education Conference (pp. 1-9). New Jersey: IEEE.
Arslanyilmaz, A., & Corpier, K. (2019). Eye tracking to evaluate comprehension of computational thinking (poster). In B. Scharlau, R. McDermott, A. Pears & M. Sabin (eds.), Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (pp. 296). New York: ACM.
Atmatzidou, S., & Demetriadis, S. (2014). How to support students’ computational thinking skills in educational robotics activities. In D. Alimisis, G. Granosik & M. Moro (eds.), Proceedings of 4th International Workshop "Teaching Robotics & Teaching with Robotics" & 5th International Conference "Robotics in Education" (pp. 43-50). Padova: TRTWR & RiE.
Atmatzidou, S., & Demetriadis, S. (2016). Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences. Robotics and Autonomous Systems, 75(B), 661-670.
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community. ACM InRoads, 2(1), 48-54.
Basawapatna, Α., Koh, Κ.Η., & Repenning, Α. (2010). Using Scalable Game Design to teach computer science from middle school to graduate students. In R. Ayfer. J. Impagliazzo & C. Laxer (eds.), ITiCSE '10: Proceedings of the fifteenth annual conference on Innovation and technology in computer science education (pp. 224-228). New York: ACM.
Basawapatna, A., Repenning, A., & Koh, K.H. (2015). Closing the cyberlearning loop: Enabling teachers to formatively assess student programming projects. Proceedings of the 46th ACM Technical Symposium on Computer Science Education SIGCSE '15 (pp. 12-17). Kansas City: ACM.
Basso, D., Fronza, I., Colombi, A., & Pahl, C. (2018). Improving assessment of computational thinking through a comprehensive framework. In M. Joy & P. Ihantola (eds.), Proceedings of the 18th Koli Calling International Conference on Computing Education Research (pp. 1-5). New York: ACM.
Basu, S. (2019). Using Rubrics Integrating Design and Coding to Assess Middle School Students' Open-ended Block-based Programming Projects. In E.K. Hawthorne, M.A. Pérez-Quiñones, S. Heckman & J. Zhang (eds.), Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 1211-1217). New York: ACM.
Basu, S., Kinnebrew, J.S., & Biswas, G. (2014). Assessing student performance in a computational-thinking based science learning environment. In S. Trausan-Matu, K.E. Boyer, M. Crosby & K. Panourgia (eds.), Intelligent Tutoring Systems (pp. 476-481). Cham: Springer.
Bell, T., & Lodi, M. (2019). Constructing computational thinking without using computers. Constructivist Foundations, 14(3), 342-351.
Bienkowski, M., Snow, E., Rutstein, D. W., & Grover, S. (2015). Assessment design patterns for computational thinking practices in secondary computer science: A first look (SRI technical report). Menlo Park: SRI International.
Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., & Engelhardt, K. (2016). Developing computational thinking in compulsory education – Implications for policy and practice. Retrieved 9 September 2018, from https://publications.jrc.ec.europa.eu/repository/bitstream/JRC104188/jrc104188_computhinkreport.pdf.
Brasiel, S., Close, K., Jeong, S., Lawanto, K., Janisiewicz, P., & Martin, T. (2017). Measuring Computational Thinking Development with the FUN! Tool. In P.J. Rich & C.B. Hodges (eds.), Emerging Research, Practice, and Policy on Computational Thinking, Educational Communications and Technology: Issues and Innovations (pp. 327-347). Cham: Springer.
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In A.F. Ball & C.A. Tyson (eds.), Proceedings of the 2012 Annual Meeting of the American Educational Research Association (pp. 1-25). Washington: AERA.
Bryman, A. (2008). Social Research Methods (3rd edition). New York: Oxford University Press.
Carnegie Mellon (n.d.). Center for Computational Thinking. Retrieved 9 September 2018, from https://www.cs.cmu.edu/~CompThink.
Chang, Z., Sun, Y., Wu, T., & Guizani, M. (2018). Scratch Analysis Tool (SAT): A modern scratch project analysis tool based on ANTLR to assess computational thinking skills. In M. Gerla, G. Hadjichristofi, C. Chrysostomou & M. Guizani (eds.), Proceedings of the 14th International Wireless Communications & Mobile Computing Conference (pp. 950-955). New Jersey: IEEE.
Chen G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162-175.
Computational Thinking with Scratch (n.d.). Computational thinking. Retrieved 9 September 2018, from http://scratched.gse.harvard.edu/ct/defining.html.
Computing at School (n.d.). Retrieved 11 November 2019, from https://www.computingatschool.org.uk.
Computing at School (2015). Computational Thinking. A guide for Teachers. Retrieved 11 November 2019, from https://community.computingatschool.org.uk/files/8550/original.pdf.
CS Unplugged at Mines (n.d.). CS Unplugged. Retrieved 9 September 2018, from http://csunplugged.mines.edu/index.html.
Curasma, R.P., Jara, N.J., Curasma, H.P., & Ornetta, V.C. (2019). Assessment of Computational Thinking in regular basic education: case I.E.T.P. “José Obrero”. In C. Gallegos & C. Silva (eds.), INTERCON 2019: IEEE XXVI International Conference on Electronics, Electrical Engineering and Computing (pp. 1-4). New Jersey: IEEE.
Cutumisu, M., Adams, C., & Lu, C. (2019). A Scoping Review of Empirical Research on Recent Computational Thinking Assessments. Journal of Science Education and Technology, 28, 651–676.
Dagienė, V., & Sentance, S. (2016). It’s Computational Thinking! Bebras tasks in the curriculum. In A. Brodnik & F. Tort (eds.), Informatics in Schools: Improvement of Informatics Knowledge and Perception (pp. 28-39). Cham: Springer.
Dagiene, V., & Stupuriene, G. (2016). Bebras – A sustainable community building model for the concept based learning of informatics and computational thinking. Informatics in Education, 15(1), 25-44.
Djambong, T., Freiman, V, Gauvin, S., Paquet, M., & Chiasson, M. (2018). Measurement of Computational Thinking in K-12 education: The need for innovative practices. In D. Sampson, D. Ifenthaler, J. Spector & P. Isaías (eds.), Digital Technologies: Sustainable Innovations for Improving Teaching and Learning (pp. 193-222). Cham: Springer.
Durak, H.Y., & Saritepeci, M. (2017). Analysis of the relation between computational thinking skills and various variables with ste structural equation model. Computers & Education, 116, 191-202.
European Commission (2016). The Computational Thinking Study. Retrieved 9 September 2018, from https://ec.europa.eu/jrc/en/computational-thinking.
Fessakis, G., Komis, V., Mavroudi, E., & Prantsoudi, S. (2018). Exploring the scope and the conceptualization of Computational Thinking at the K-12 classroom level curriculum. In M.S. Khine (ed.), Computational Thinking in the STEM Disciplines: Foundations and Research Highlights (pp. 181-212). Cham: Springer.
Fessakis, G., Mavroudi, E., & Prantsoudi, S. (2017). Current Conception of Computational Thinking at the K-12 classroom level curriculum (poster). In V. Dagienė & A. Hellas (eds.), Informatics in Schools: Focus on Learning Programming (pp. 1-6). Cham: Springer.
Giordano, D., Maiorana, F., Csizmadia, A., Marsden, S., Riedesel, C., Mishra, S., & Vinikiene, L. (2015). New horizons in the assessment of computer science at school and beyond: Leveraging on the ViVA platform. In N. Ragonis & P. Kinnunen (eds.), Proceedings of the 2015 ITiCSE on Working Group Reports (pp. 117-147). New York: ACM.
Gouws, L., Bradshaw, K., & Wentworth, P. (2013). First year student performance in a test for Computational Thinking. In J. McNeill, K. Bradshaw, P. Machanick & M. Tsietsi (eds.), Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference (pp. 271-277). New York: ACM.
Grover, S. (2017). Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom. In P.J. Rich & C.B. Hodges (eds.), Emerging Research, Practice, and Emerging Research, Practice, and Policy on Computational Thinking. Educational Communications and Technology: Issues and Innovations (pp. 269-288). Cham: Springer.
Grover, S., & Basu, S. (2017). Measuring Student Learning in Introductory Block-Based Programming: Examining Misconceptions of Loops, Variables, and Boolean Logic. In M.E. Caspersen, S.H. Edwards, T. Barnes & D.D.Garcia (eds.), Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (pp. 267-272). New York: ACM.
Grover, S., Basu, S., Bienkowski, M., Eagle, M., Diana, N., & Stamper, J. (2017). A framework for using hypothesis-driven approaches to support data-driven learning analytics in measuring computational thinking in block-based programming environments. ACM Transactions on Computing Education, 17(3), 1-25.
Grover, S., Cooper, S., & Pea, R. (2014). Assessing computational learning in K-12. In Å. Cajander, M. Daniels, T. Clear & A.N. Pears (eds.), Proceedings of the 2014 Conference on Innovation & technology in computer science education (pp. 57-62). New York: ACM.
Grover, S., & Pea, R. (2013). Computational Thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.
Grover, S., Pea, R., & Cooper, S. (2015). “Systems of Assessments” for deeper learning of Computational Thinking in K-12. In J.E. King & B.M. Gordon (eds.), Proceedings of the 2015 Annual Meeting of the American Educational Research Association (pp. 1-10). Washington: AERA.
Hadad, R., Thomas, K., Kachovska, M., & Yin, Y. (2019). Practicing formative assessment for computational thinking in making environments. Journal of Science Education and Technology, 29, 162–173.
Heintz, F., Mannila, L., & Färnqvist, T. (2016). A review of models for introducing computational thinking, computer science and computing in K-12 education. In S. Frezza, D. Onipede, K. Vernaza & M. Ford (eds.), 2016 IEEE Frontiers in Education Conference (pp. 1-9). New Jersey: IEEE.
Hoover, A.K., Barnes, J., Fatehi, B., Moreno-León, J., Puttick, G., Tucker-Raymond, E., & Harteveld, C. (2016). Assessing Computational Thinking in students' game designs. In J. Kaye et al. (eds.), Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 173-179). New York: ACM.
ISTE, & CSTA (2011). Operational definition of Computational Thinking in K-12 education. Retrieved 9 September 2018, from https://id.iste.org/docs/ct-documents/computational-thinking-operational-definition-flyer.pdf?sfvrsn=2.
ISTE, & CSTA (2015). Computational thinking leadership toolkit. Retrieved 9 September 2018, from https://id.iste.org/docs/ct-documents/ct-leadershipt-toolkit.pdf?sfvrsn=4.
Kalelioglu, F., Gulbahar, Y. & Kukul, V. (2016). A framework for computational thinking based on a systematic research review. Baltic J. Modern Computing, 4(3), 583-596.
Koh, K.H., Basawapatna, A., Bennett, V., & Repenning, A. (2010). Towards the automatic recognition of computational thinking for adaptive visual language learning. In C. Hundhausen, E. Pietriga, P. Díaz & M.B. Rosson (eds.), Proceedings of the 2010 IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 59-66). New Jersey: IEEE.
Koh, K.H., Basawapatna, A., Nickerson, H., & Repenning, A. (2014). Real Time assessment of computational thinking. In S. Fleming, A. Fish & C. Scaffidi (eds.), Proceedings of the 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 49-52). New Jersey: ΙΕΕΕ.
Koh, K.H., Nickerson, H., Basawapatna, A., & Repenning A. (2014). Early validation of Computational thinking pattern analysis. In Å. Cajander, M. Daniels, T. Clear & A.N. Pears (eds.), Proceedings of the 2014 conference on Innovation & technology in computer science education (pp. 213-218). New York: ACM.
Korkmaz, Ö., Cakir, R., & Yasar Özden, M. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558-569.
Leifheit, L., Tsarava, K., Moeller, K., Ostermann, K., Golle, J., Trautwein, U., & Ninaus, M. (2019). Development of a Questionnaire on Self-concept, Motivational Beliefs, and Attitude Towards Programming. Q. Cutts & T. Brinda (eds.), Proceedings of the 14th Workshop in Primary and Secondary Computing Education (pp. 1-9). New York: ACM.
Lye, S.Y., & Koh, J.H.L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12?. Computers in Human Behavior, 41, 51-61.
Lytle, N., Catete, V., Dong, Y., Boulden, D., Akram, B., Houchins, J., Barnes, T., & Wiebe, E. (2019). CEO: A triangulated evaluation of a modeling-based CT-infused CS activity for non-CS middle grade students. In M. Zhang, B. Yang, S. Cooper & A. Luxton-Reilly (eds.), Proceedings of the ACM Conference on Global Computing Education (pp. 58-64). New York: ACM.
Maiorana, F., Giordano, D., & Morelli, R. (2015). Quizly: A live coding assessment platform for App Inventor. In E. Kraemer, C. Ermel & S. Fleming (eds.), Proceedings of the 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (Blocks and Beyond workshop) (pp. 25-30). New Jersey: IEEE.
Marinus, E., Powell, Z., Thornton, R., McArthur, G., & Crain, S. (2018). Unravelling the cognition of coding in 3-to-6-year olds: The development of an assessment tool and the relation between coding ability and cognitive compiling of syntax in natural language. In L. Malmi et al. (eds.), Proceedings of the 2018 ACM Conference on International Computing Education Research (pp. 133-141). New York: ACM.
Merkouris, Α., & Chorianopoulos, Κ. (2019). Programming embodied interactions with a remotely controlled educational robot. ACM Transactions on Computing Education, 19(4), 1-19.
Moreno, J., & Robles, G. (2014). Automatic detection of bad programming habits in scratch: A preliminary study. In M. Castro & E. Tovar (eds.), 2014 IEEE Frontiers in Education Conference (pp. 403-406). New Jersey: IEEE.
Moreno-León, J., & Robles, G. (2015a). Analyze your Scratch projects with Dr. Scratch and assess your Computational Thinking Skills. In C. Terosier et al. (eds.), 5th International Scratch Conference (pp. 1-7). Amsterdam: Scratch2015AMS.
Moreno-León, J., & Robles, G. (2015b). Dr. Scratch: a web tool to automatically evaluate Scratch projects. In J. Gal-Ezer, S. Sentance & J. Vahrenhold (eds.), Proceedings of the Workshop in Primary and Secondary Computing Education (pp. 1-2). New York: ACM.
Moreno-León, J., Robles, G., & Román-González, M. (2016). Comparing computational thinking development assessement scores with software complexity metrics. In M. Al-Mualla, M.E. Auer & S. Al-Samahi (eds.), Proceedings of the IEEE 2016 Global Engineering Education Conference (pp. 1040-1045). New Jersey: IEEE.
Moreno-León, J., Robles, G., & Román-González, M. (2017). Can we measure computational thinking with tools? Present and future of Dr. Scratch. In G. Robles, H. Osman, A. Chis & F. Hermans (eds.), SATToSE 2017: Seminar Series on Advanced Techniques & Tools for Software Evolution (pp. 1-5). Madrid: Universidad Rey Juan Carlos.
Moreno-León, J., Román-González, M., Harteveld, C., & Robles, G. (2017). On the automatic assessment of computational thinking skills: A Comparison with Human Experts. In G. Mark et al. (eds.), Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2788-2795). New York: ACM.
Mueller, J., Beckett, D., Hennessey, E., & Shodiev, H. (2017). Assessing computational thinking across the curriculum. In P.J. Rich & C.B. Hodges (eds.), Emerging Research, Practice, and Policy on Computational Thinking, Educational Communications and Technology: Issues and Innovations (pp. 251-267). Cham: Springer.
Mühling, A., Ruf, A., & Hubwieser, P. (2015). Design and first results of a psychometric test for measuring basic programming abilities. In J. Gal-Ezer, S. Sentance & J. Vahrenhold (eds.), Proceedings of the Workshop in Primary and Secondary Computing Education (pp. 2-10). New York: ACM.
Ota, G., Morimoto, Y., & Kato, H. (2016). Ninja code village for scratch: Function samples/function analyser and automatic assessment of computational thinking concepts. In A. Blackwell, G. Stapleton & B. Plimmer (eds.), 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 238-239). New Jersey: IEEE.
Palts, Τ., & Pedaste, Μ. (2017). Tasks for assessing skills of computational thinking. In R. Davoli, M. Goldweber, G. Rößling & I. Polycarpou (eds.), Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (pp. 367). New York: ACM.
Papavlasopoulou, S., Sharma, K., & Giannakos, M.N. (2019). Coding activities for children: Coupling eye-tracking with qualitative data to investigate gender differences. Computers in Human Behavior, 105, 1-11.
Papert, S. (1993). Mindstorms: Children, Computers, and Powerful Ideas (2nd ed.). New York: Basic Books.
Park, T.H., Kim, M.C., Chhabra, S., Lee, B., & Forte, A. (2016). Reading hierarchies in code: Assessment of a basic computational skill. In A. Clear, E. Cuadros-Vargas, J. Carter & Y. Tupac (eds.), Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (pp. 302-307). New York: ACM.
Pérez, A.D.F., & Valladares, G.M. (2018). Development and assessment of computational thinking: A methodological proposal and a support tool. In C.S. González González, M. Castro, M. Llamas Nistal (eds.), Proceedings of the 2018 IEEE Global Engineering Education Conference (pp. 787-795). New Jersey: IEEE.
Pérez-Marín, D., Hijón-Neira, R., Bacelo, A., & Pizarro, C. (2018). Can computational thinking be improved by using a methodology based on metaphors and scratch to teach computer programming to children?. Computers in Human Behavior, 105, 1-10.
Petri, G., & Gresse von Wangenheim, C. (2017). How games for computing education are evaluated? A systematic literature review. Computers & Education, 107, 68-90.
Portelance, D.J., & Bers, M.U. (2015). Code and tell: Assessing young children's learning of computational thinking using peer video interviews with ScratchJr. In M.U. Bers & G.L. Revelle (eds.), Proceedings of the 14th International Conference on Interaction Design and Children (pp. 271-274). New York: ACM.
Rich, P.J., Egan, G., & Ellsworth, J. (2019). A framework for decomposition in Computational Thinking. In B. Scharlau, R. McDermott, A. Pears & M. Sabin (eds.), Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (pp. 416-421). New York: ACM.
Rodriguez, B., Kennicutt, S., Rader, C., & Camp, T. (2017). Assessing Computational Thinking in CS unplugged activities. In M.E. Caspersen, S.H. Edwards, T. Barnes & D.D. Garcia (eds.), Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (pp. 501-506). New York: ACM.
Rodriguez, B., Rader, C., & Camp, T. (2016). Using student performance to assess CS unplugged activities in a classroom environment. In A. Clear, E. Cuadros-Vargas, J. Carter & Y. Tupac (eds.), Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (pp. 95-100). New York: ACM.
Rojas Lopez, A., & Garcia-Penalvo, F. (2016). Relationship of knowledge to learn in programming methodology and evaluation of computational thinking. In F.J. García-Peñalvo (ed.), Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 73-77). New York: ACM.
Román-González, M., Moreno-Leon, J., & Robles, G. (2017). Complementary tools for computational thinking assessment. In S.C. Kong, J. Sheldon & K.Y. Li (eds.), Proceedings of the International Conference on Computational Thinking Education (pp. 154-159). Hong Kong: The Education University of Hong Kong.
Román-González, M., Moreno-Leon, J., & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In S.C. Kong & H. Abelson (eds.), Computational Thinking Education (pp.79-98). Singapore: Springer.
Román-González, M., Pérez-González, J., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior, 72, 678-691.
Román-González, M., Pérez-González, J., Moreno-Leon, J., & Robles, G. (2016). Does computational thinking correlate with personality?: The non-cognitive side of computational thinking. In F.J. García-Peñalvo (ed.), Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 51-58). New York: ACM.
Román-González, M., Pérez-González, J., Moreno-Leon, J., & Robles, G. (2018a). Extending the nomological network of computational thinking with non-cognitive factors. Computers in Human Behavior, 80, 441-459.
Román-González, M., Pérez-González, J.C., Moreno-Leon, J. & Robles, G. (2018b). Can computational talent be detected? Predictive validity of the Computational Thinking Test. International Journal of Child-Computer Interaction, 18, 47–58.
Rowe, E., Asbell-Clarke, J., Cunningham, K., & Gasca, S. (2017a). Assessing implicit computational thinking in Zoombinis gameplay. In S. Deterding et al. (eds.), Proceedings of the 12th International Conference on the Foundations of Digital Games (pp. 1-4). New York: ACM.
Rowe, E., Asbell-Clarke, J., Cunningham, K., & Gasca, S. (2017b). Assessing Implicit Computational Thinking in Zoombinis Gameplay: Pizza Pass, Fleens & Bubblewonder Abyss. In B. Schouten et al. (eds.), CHI PLAY '17 Extended Abstracts: Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play (pp. 195-200). New York: ACM.
Royal Society (2012). Shut down or restart: The way forward for computing in UK schools. Retrieved 9 September 2018, from https://royalsociety.org/~/media/Royal_Society_Content/education/policy/computing-in-schools/2012-01-12-Computing-in-Schools.pdf.
Salac, J. (2019). Personalized Assessment Worksheets for Scratch (PAWS): Exploring a bridge between interviews, written assessments, and artifact analysis. In McCartney et al. (eds.), Proceedings of the 2019 ACM Conference on International Computing Education Research (pp. 351-352). New York: ACM.
Santisteban, J., & Santisteban-Munoz, J. (2018). Psychometric Computational Thinking Test. In I. Polycarpou, J.C. Read, P.G. Andreou & M. Armoni (eds.), Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (p. 393). New York: ACM. https://doi.org/10.1145/3197091.3205823.
Seiter, L., & Foreman, B. (2013). Modeling the Learning Progressions of Computational Thinking of Primary Grade Students. In B. Simon, A. Clear & Q.I. Cutts (eds.), Proceedings of the ninth Annual International ACM Conference on International Computing Education Research (pp. 59-66). New York: ACM.
Selby, C., & Woollard, J. (2013). Computational thinking: the developing definition. University of Southampton. Retrieved 12 September 2018, from https://eprints.soton.ac.uk/356481.
Sherman, M., & Martin, F. (2015). The assessment of mobile computational thinking. Journal of Computing Sciences in Colleges, 30(6), 53-59.
Snow, E., Rutstein, D., Bienkowski, M., & Xu, Y. (2017). Principled assessment of student learning in high school computer science. In J. Tenenberg et al. (eds.), Proceedings of the 2017 ACM Conference on International Computing Education Research (pp. 209-216). New York: ACM.
Srinivas, M.J., Roy, M.M, Sagri, J.N., & Kumar, V. (2018). Assessing Scratch programmers’ development of computational thinking with transaction-level data. In S. Chakraverty, A. Goel & S. Misra (eds.), Towards Extensible and Adaptable Methods in Computing (pp. 399-407). Singapore: Springer.
Swanson, H., Anton, G., Bain, C., Horn, M., & Wilensky, U. (2019). Introducing and assessing computational thinking in the secondary science classroom. In S.C. Kong & H. Abelson (eds.), Computational Thinking Education (pp. 99-117). Singapore: Springer.
Teaching London Computing (n.d.). Teaching London Computing: A resource hub from CAS London and CS4FN. Retrieved 20 August 2019, from https://teachinglondoncomputing.org.
Troiano, G.M., Snodgrass, S., Argimak, E., Robles, G., Smith. G., Cassidy, M., Tucker-Raymond, E., Puttick, G., & Harteveld, C. (2019). Is my game OK Dr. Scratch?: Exploring programming and computational thinking development via metrics in student-designed serious games for STEM. In J.A. Fails (ed.), Proceedings of the 18th ACM International Conference on Interaction Design and Children (pp. 208-219). New York: ACM.
Tsarava, K., Leifheit, L., Ninaus, M., Román-González, M., Butz, M.V., Golle, J., Trautwein, U., & Moeller, K. (2019). Cognitive correlates of Computational Thinking: Evaluation of a blended unplugged/plugged-In course. In Q. Cutts & T. Brinda (eds.), WiPSCE'19: Proceedings of the 14th Workshop in Primary and Secondary Computing Education (pp. 1-9). New York: ACM.
University of Canterbury (n.d.). CS Education Research Group: Computer Science Unplugged. Retrieved 9 September 2018, from https://csunplugged.org/en.
Vilniaus Universitetas (n.d.). Bebras - International Challenge on Informatics and Computational Thinking. Retrieved 9 September, 2018, from https://www.bebras.org.
Webb, D.C. (2010). Troubleshooting assessment: an authentic problem solving activity for it education. Procedia - Social and Behavioral Sciences, 9, 903-907.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Trouille, L., Jona, K., & Wilensky, U. (2014). Interactive assessment tools for computational thinking in high school STEM classrooms. INTETAIN 2014, Springer LNICST, 136, 22-25.
Weintrop, D., & Wilensky, U. (2015). Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs. In B. Dorn, J. Sheard & Q.I. Cutts (eds.), Proceedings of the Eleventh Annual International Conference on International Computing Education Research (pp. 101-110). New York: ACM.
Werner, L., Denner, J., & Campe, S. (2015). Children programming games: A strategy for measuring computational learning. ACM Transactions on Computing Education, 14(4), 1-22.
Werner, L., Denner, J., Campe, S., & Kawamoto, D.C. (2012). The fairy performance assessment: Measuring computational thinking in middle school. In L.A. Smith King, D.R. Musicant, T. Camp & P.T. Tymann (eds.), Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (pp. 215-220). New York: ACM.
Wiebe, E., London, J., Aksit, O., Mott, B.W., Boyer, K.E., & Lester, J.C. (2019). Development of a lean computational thinking abilities assessment for middle grades students. In E.K. Hawthorne, M.A. Pérez-Quiñones, S. Heckman & J. Zhang (eds.), Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 456-461). New York: ACM.
Wilson, A., Hainey, T., & Connolly, T. (2012). Evaluation of computer games developed by primary school children to gauge understanding of programming concepts. In P. Felicia (ed.), 6th European Conference on Games Based Learning (pp. 549-558). Cork: ACPL.
Wing, J.M. (2006), Computational thinking. Communications of the ACM, 49(3), 33–35.
Wing, J.M. (2011). Research Notebook: Computational Thinking - What and Why?. The Link. Retrieved 9 September 2018, from https://www.cs.cmu.edu/link/research-notebook-computational-thinking-what-and-why.
Yadav, A., Burkhart, D., Moix, D., Snow, E., Bandaru, P., & Clayborn, L. (2015). Sowing the Seeds: A Landscape Study on Assessment in Secondary Computer Science Education. New York: CSTA.
Yagci, M. (2019). A valid and reliable tool for examining computational thinking skills. Education and Information Technologies, 24, 929-951.
Zhang, L., & Nouri, J. (2019). A systematic review of learning computational thinking through Scratch in K-9. Computers & Education, 141, 1-25.
Zhang, N., & Biswas, G. (2019). Defining and assessing students’ computational thinking in a learning by modeling environment. In S.C. Kong & H. Abelson (eds.), Computational Thinking Education (pp. 203-221). Singapore: Springer.