Evaluating 2D and 3D serious games: The significance of student-player characteristics
Published:
Dec 17, 2019
Keywords:
2D game 3D game mental rotation playing experience spatial cognition user experience quantitative approach
Abstract
Serious games are gaining ground in all educational levels. Important aspects of their implementation in education are related to their features, their relevance to pedagogy and the learners' views. This quantitative research focused on characteristics of pedagogical departments’ students (gender, scientific background, prior gaming and ICT skills) and factors are considered to affect and shape their learning experience when playing 2D and 3D serious games (immersion, enjoyment, perceived usefulness-knowledge improvement, perceived narratives' adequacy, perceived realism, perceived feedback's adequacy, perceived audiovisual adequacy, perceived relevance to personal interests, perceived goal's clarity, perceived ease of use, adequacy of the learning material and motivation). A total of 542 university students participated in the study. The data analysis revealed that they preferred the 2D game. Their gender, scientific background and ICT skills did not have an impact on their views. On the other hand, students with high game-playing competence seem to have a better learning experience, as their scores in most of the factors were higher compared to the ones with lower game-playing competence. Those results led us to consider other aspects of learners beyond ordinary demographics such as self-regulation, spatial cognition and mental rotation and to examine the evidence that serious games can improve these skills.
Article Details
- How to Cite
-
Kaimara, P., Fokides, E., Oikonomou, A., Atsikpasi, P., & Deliyannis, I. (2019). Evaluating 2D and 3D serious games: The significance of student-player characteristics. Dialogoi! Theory and Praxis in Education, 5, 36–56. https://doi.org/10.12681/dial.20704
- Issue
- Vol. 5 (2019)
- Section
- Scientific columns
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution Non-Commercial License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g. post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (preferably in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Downloads
Download data is not yet available.
References
Abt C. C. (1970). Serious games. New York: Viking Press. https://doi.org/10.1177/000276427001400113
Almeida, P., Abreu, J., Silva, T., Varsori, E., Oliveira, E., Velhinho, A., … Oliveira, D. (2018). Iterative user experience evaluation of a user interface for the unification of tv contents. Communications in Computer and Information Science, 813, 44-57. https://doi.org/10.1007/978-3-319-90170-1_4
Anderson, S. J., Jamniczky, H., Williams, C., Krigolson, O., Coderre, S., & Hecker, K. (2019). Learning from Two-Dimensional (2D) versus Three-Dimensional Anatomical Models: Assessing Working Memory Requirements Using Electroencephalography (EEG). The FASEB Journal, 33(1_supplement), 328-5.
Bernhaupt, R. (2010). User Experience Evaluation in Entertainment. In R. Bernhaupt (Ed.), Evaluating user experience in games: Concepts and methods (pp. 3-7). London: Springer. https://doi.org/10.1007/978-1-84882-963-3_1
Bernhaupt, R., & Mueller, F. F. (2016, May). Game user experience evaluation. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 940-943. ACM. https://doi.org/10.1145/2851581.2856683
Bevan, N. (2001). International standards for HCI and usability. International journal of human-computer studies, 55(4), 533-552. https://doi.org/10.1006/ijhc.2001.0483
Buck, R., Khan, M., Fagan, M., & Coman, E. (2018). The User Affective Experience Scale: A measure of emotions anticipated in response to pop-up computer warnings. International Journal of Human-Computer Interaction, 34(1), 25-34. https://doi.org/10.1080/10447318.2017.1314612
Cai, Y., Goei, S. L., & Trooster, W. (Eds.) (2016). Simulation and serious games for education. Singapore: Springer. https://doi.org/10.1007/978-981-10-0861-0
Charlesworth, P., Drummer, T., Hungwe, K., & Sorby, S. (2005, June). Developing 3D spatial visualization skills for non engineering students. Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition, 10.428.1-10.428.11. American Society for Engineering Education.
Chung, L. Y., & Chang, R. C. (2017). The effect of gender on motivation and student achievement in digital game-based learning: A case study of a contented-based classroom. Eurasia Journal of Mathematics, Science and Technology Education, 13(6), 2309-2327. https://doi.org/10.12973/eurasia.2017.01227a
Cohen, J. (2013). Statistical power analysis for the behavioral sciences. New York: Routledge
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661-686. https://doi.org/10.1016/j.compedu.2012.03.004
Corder, G. W., & Foreman, D. I. (2009). Nonparametric statistics for non-statisticians: Α step-by-step approach. John Wiley & Sons.
Csikszentmihalyi, M., (1975). Beyond boredom and anxiety. San Francisco: Jossey-Bass. Retrieved from http://psy2.ucsd.edu/~nchristenfeld/Happiness_Readings_files/Class%207%20-%20Csikszentmihalyi%201975.pdf
Dawson, C. (2019). Tackling Limited Spatial Ability: Lowering One Barrier into STEM?. European Journal of Science and Mathematics Education, 7(1), 14-31. Retrieved from https://files.eric.ed.gov/fulltext/EJ1203669.pdf
de Freitas, S. (2018). Are games effective learning tools? A review of educational games. Journal of Educational Technology & Society, 21(2), 74-84. Retrieved from https://psycnet.apa.org/record/2018-18436-006
Deliyannis, I., & Kaimara, P. (2019). Developing Smart Learning Environments Using Gamification Techniques and Video Game Technologies. Didactics of Smart Pedagogy, 285-307. https://doi.org/10.1007/978-3-030-01551-0_15
DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Sage publications.
Dindar, M. (2018). An empirical study on gender, video game play, academic success and complex problem solving skills. Computers & Education, 125, 39-52. https://doi.org/10.1016/j.compedu.2018.05.018
Erfani, M., El-Nasr, M. S., Milam, D., Aghabeigi, B., Lameman, B. A., Riecke, B. E., Maygoli, H., & Mah, S. (2010). The effect of age, gender, and previous gaming experience on game play performance. In P. Forbrig, F. Paternó, A. Mark Pejtersen (Eds.), Human-Computer Interaction. HCIS 2010. IFIP Advances in Information and Communication Technology, vol 332. Berlin: Springer. https://doi.org/10.1007/978-3-642-15231-3_33
Fokides, E., Kaimara, P., Deliyannis, I., & Atsikpasi, P. (2019). Development of a scale for measuring the learning experience in serious games. Preliminary results. Proceedings of 1st International Conference on Digital Culture & AudioVisual Challenges, Department of Audiovisual Arts, Ionian University, Corfu June 1-2, 2018 (in press).
Fokides, E., Atsikpasi, P., Kaimara, P., & Deliyannis, I. (2019a). Factors influencing the subjective learning effectiveness of serious games. Journal of Information Technology Education: Research, 18, 437-466. https://doi.org/10.28945/4441
Fokides, E., Atsikpasi, P., Kaimara, P., & Deliyannis, I. (2019b). Let players evaluate serious games. Design and validation of the Serious Games Evaluation Scale. International Computer Games Association Journal, 31(3), 1-22. https://doi.org/10.3233/ICG-190111
Fokides, E. (2017). Pre-service teachers' intention to use MUVES as practitioners. A structural equation modeling approach. Journal of Information Technology Education, 16, 47-68. https://doi.org/10.28945/3645
Franzwa, C., Tang, Y., Johnson, A., & Bielefeldt, T. (2014). Balancing fun and learning in a serious game design. International Journal of Game-Based Learning (IJGBL), 4(4), 37-57.
Garmen, P., Rodríguez, C., García-Redondo, P., & San-Pedro-Veledo, J. C. (2019). Multiple intelligences and video games: Assessment and intervention with TOI software. Comunicar: Media Education Research Journal, 27(58), 95-104. https://doi.org/10.1111/j.1365-2729.2012.00489.x
Gecu, Z., & Cagiltay, K. (2015). Mental Rotation Ability and Computer Game Experience. International Journal of Game-Based Learning (IJGBL), 5(4), 15-26. https://doi.org/10.4018/ijgbl.2015100102
Gee, J. P. (2003). What video games have to teach us about learning and literacy. Computers in Entertainment, 1(1), 20-20. https://doi.org/10.1145/950566.950595
Hassenzahl, M. (2008). User experience (UX): towards an experiential perspective on product quality. In Proceedings of the 20th international Conference of the Association Francophone D'interaction Homme-Machine, 11-15. ACM. https://doi.org/10.1145/1512714.1512717
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016). A systematic literature review of games-based learning empirical evidence in primary education. Computers & Education, 102, 202-223. https://doi.org/10.1016/j.compedu.2016.09.001
Hu, F., & Liu, Y. (2010, October). Impact of experience and gender differences on users' perceptions on mobile game. Proceedings of the Multimedia Technology (ICMT), 2010 International Conference, 1-2. IEEE. https://doi.org/10.1109/icmult.2010.5629762
Kaimara, P., Deliyannis, I., Papadopoulou, A., Oikonomou, A., & Fokides, E. (2018). Smart Education: Τα ψηφιακά παιχνίδια ως κίνητρο για μάθηση [Smart Education: Digital games as motivation for learning]. Proceedings of the 2nd International Experiential Conference on Applied Teaching, 385-391. Drama, Greece: Educational Circle.
Kaimara, P., & Deliyannis, I. (2019). Why should I play this game? The role of motivation in Smart Pedagogy. In L. Daniela (Ed.), Didactics of Smart Pedagogy (pp. 113-137). Springer, Cham. https://doi.org/10.1007/978-3-030-01551-0_6
Kaimara, P., Fokides, E., Plerou, A., Atsikpasi, P., & Deliyannis, I. (2020). Serious games effect analysis on player’s characteristics. International Journal of Smart Education and Urban Society, 11(1), 75-91. https://doi.org/10.4018/IJSEUS.2020010106
Keller, J. M. (2010). Five fundamental requirements for motivation and volition in technology-assisted distributed learning environments. Revista Inter Ação, 35(2), 305-322. https://doi.org/10.5216/ia.v35i2.12668
Kirkley, S. E., Tomblin, S., & Kirkley, J. (2005). Instructional design authoring support for the development of serious games and mixed reality training. Proceedings of the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC). Retrieved from https://pdfs.semanticscholar.org/abeb/00c31a0cbc8920c37493f7922b95f333bc4c.pdf
Koeffel, C., Hochleitner, W., Leitner, J., Haller, M., Geven, A., & Tscheligi, M. (2010). Using heuristics to evaluate the overall user experience of video games and advanced interaction games. In R. Bernhaupt (Ed.), Evaluating user experience in games: Concepts and Methods (pp. 233-256). London: Springer. https://doi.org/10.1007/978-1-84882-963-3_13
Lallemand, C. (2015). Towards consolidated methods for the design and evaluation of user experience (Doctoral dissertation). University of Luxembourg, Luxembourg. Retrieved from http://orbilu.uni.lu/handle/10993/21463
Lallemand, C., Gronier, G., & Koenig, V. (2015). User experience: A concept without consensus? Exploring practitioners' perspectives through an international survey. Computers in Human Behavior, 43, 35-48. https://doi.org/10.1016/j.chb.2014.10.048
Law, E. L. C., & Sun, X. (2012). Evaluating user experience of adaptive digital educational games with Activity Theory. International Journal of Human-Computer Studies, 70(7), 478-497. https://doi.org/10.1016/j.ijhcs.2012.01.007
Law, E. L. C., Roto, V., Hassenzahl, M., Vermeeren, A. P., & Kort, J. (2009, April). Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 719-728). ACM. https://doi.org/10.1145/1518701.1518813
Levine, S. C., Foley, A., Lourenco, S., Ehrlich, S., & Ratliff, K. (2016). Sex differences in spatial cognition: Advancing the conversation. Wiley Interdisciplinary Reviews: Cognitive Science, 7(2), 127-155. https://doi.org/10.1002/wcs.1380
Mayer, R. E. (2016). What should be the role of computer games in education? Policy Insights from the Behavioral and Brain Sciences, 3(1), 20-26. https://doi.org/10.1177/2372732215621311
Mayer, R. E. (2019). Computer games in education. Annual review of Psychology, 70, 531-549. https://doi.org/10.1146/annurev-psych-010418-102744
Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., van Ruijven, T., Lo, J., Kortmann, R., & Wenzler, I. (2014). The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Technology, 45(3), 502-527. https://doi.org/10.1111/bjet.12067
McMahan, A. (2003). Immersion, engagement, and presence: A method for analyzing 3-D video games. In M. J. P. Wolf & P. Bernard (Eds.), The video game theory reader (pp. 67–86). New York, NY: Routledge. Retrieved from https://alisonmcmahan.com/sites/default/files/pdfs/papers/ch3-McMahanrev.pdf
Moreau, D., Mansy-Dannay, A., Clerc, J., & Guerrien, A. (2010). Academic program and mental rotation performance: Evidence for a developmental effect on individual differences in early adulthood. Education Sciences & Psychology, 17(2), 21-28.
Nacke, L. E., Drachen, A., & Göbel, S. (2010). Methods for evaluating gameplay experience in a serious gaming context. International Journal of Computer Science in Sport, 9(2), 1-12. Retrieved from http://iacss.org/fileadmin/user_upload/IJCSS_FullPaper/Vol9_Ed2/IJCSS-Volume9_Edition2_4_nacke.pdf
O'Brien, S. (2016, June). Why User Experience Design is Critical to Driving and Maintaining User Engagement and Motivation for Online and Mobile Educational Tools. Proceedings of the Future of Education Conference (p. 328). Padova, Italy: Libreriauniversitaria.iit Edizioni Retrieved from https://conference.pixel-online.net/FOE/files/foe/ed0006/FP/2819-ENT1831-FP-FOE6.pdf
Papastergiou, M. (2009). Digital game-based learning in high school computer science education. Computers & Education, 52(1), 1-12. https://doi.org/10.1016/j.compedu.2008.06.004
Peters, M., Lehmann, W., Takahira, S., Takeuchi, Y., & Jordan, K. (2006). Mental rotation test performance in four cross-cultural samples (n= 3367): overall sex differences and the role of academic program in performance. Cortex, 42(7), 1005-1014. https://doi.org/10.1016/s0010-9452(08)70206-5
Phan, M. H., Keebler J. R., & Chaparro, B. S. (2016). The Development and Validation of the Game User Experience Satisfaction Scale (GUESS). Human factors, 58(8), 1217-1247. https://doi.org/10.1177/0018720816669646
Plass, J. L., Homer, B. D., MacNamara, A., Ober, T., Rose, M. C., Pawar, S., ... & Olsen, A. (2019). Emotional design for digital games for learning: The effect of expression, color, shape, and dimensionality on the affective quality of game characters. Learning and Instruction. https://doi.org/10.1016/j.learninstruc.2019.01.005
Raptis, G. E., Fidas, C. A., & Avouris, N. M. (2016, October). Do field dependence-independence differences of game players affect performance and behaviour in cultural heritage games? Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (pp. 38-43). ACM. https://doi.org/10.1145/2967934.2968107
Sánchez, J. G., Zea, N. P., & Gutiérrez, F. L. (2009, August). Playability: How to identify the player experience in a video game. Proceedings of the IFIP Conference on Human-Computer Interaction, 356-359. Berlin: Springer. https://doi.org/10.1007/978-3-642-03655-2_39
Siegel, S., & Castellan Jr, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.
Sims, R. (2000). An interactive conundrum: Constructs of interactivity and learning theory. Australasian Journal of Educational Technology, 16(1). https://doi.org/10.14742/ajet.1821
Spence, I., & Feng, J. (2010). Video games and spatial cognition. Review of General Psychology, 14(2), 92-104. https://doi.org/10.1037/a0019491
Spieler, B., & Slany, W. (2018). Game development-based learning experience: Gender differences in game design. arXiv preprint arXiv:1805.04457
Statista: The Statistics Portal (2018). Retrieved from https://www.statista.com/statistics/232383/gender-split-of-us-computer-and-video-gamers
Tiede, J., & Grafe, S. (2018, September). Piloting Two Educational Games in Five European Countries: Teachers' Perceptions of Student Motivation and Classroom Engagement. In 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) (pp. 1-8). IEEE. https://doi.org/10.1109/vs-games.2018.8493438
Tobias, S., Fletcher, J. D., & Wind, A. P. (2014). Game-based learning. Ιn M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 485-503). New York, NY: Springer.
Yurt, E., & Tünkler, V. (2016). A Study on the Spatial Abilities of Prospective Social Studies Teachers: A Mixed Method Research. Educational Sciences: Theory and Practice, 16(3), 965-986. https://doi.org/10.12738/estp.2016.3.0324
Van Eck, R. N., Shute, V. J., & Rieber, L. P. (2017). Leveling up: Game design research and practice for instructional designers. In R. Reiser & J. Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 227-285). Upper Saddle River: Pearson Education. Retrieved from http://myweb.fsu.edu/vshute/pdf/levelingup.pdf
Veltri, N. F., Krasnova, H., Baumann, A., and Kalayamthanam, N. (2014). Gender differences in online gaming: a literature review. Proceedings of the 20th Americas Conference on Information Systems. Boston, MA: AMCIS. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.667.4530&rep=rep1&type=pdf
Wang, Y., Rajan, P., Sankar, C. S., & Raju, P. K. (2016). Let them play: the impact of mechanics and dynamics of a serious game on student perceptions of learning engagement. IEEE Transactions on Learning Technologies, 10(4), 514-525. https://doi.org/10.1109/tlt.2016.2639019
Wasserman, J. A., & Rittenour, C. E. (2019). Who wants to play? Cueing perceived sex-based stereotypes of games. Computers in Human Behavior, 91, 252-262. https://doi.org/10.1016/j.chb.2018.09.003
Westera, W. (2019). Why and How Serious Games can Become Far More Effective: Accommodating Productive Learning Experiences, Learner Motivation and the Monitoring of Learning Gains. Educational Technology & Society, 22(1), 59–69. Retrieved from https://www.j-ets.net/ets/journals/22_1/5.pdf
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of educational psychology, 81(3), 329-339. https://doi.org/10.1037//0022-0663.81.3.329
Zhonggen, Y. (2019). A Meta-Analysis of Use of Serious Games in Education over a Decade. International Journal of Computer Games Technology, 2019. https://doi.org/10.1155/2019/4797032
