UTILITARIAN EVALUATION OF EDUCATIONAL GAMES IN POST-SECONDARY STUDENTS by Nicoline Turner B.Sc., University of Northern British Columbia, 2011 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN BUSINESS ADMINISTRATION UNIVERSITY OF NORTHERN BRITISH COLUMBIA July 2025 © Nicoline Turner, 2025 ii Abstract Educational games have become a niche genre, with few games produced for the postsecondary market. This study intended to investigate game mechanics by providing an educational game for first-year university math courses. Participants displayed such an aversion to the educational game genre that the study shifted to a qualitative investigation of students’ attitudes toward educational games. Interviews revealed that math aversion was a more powerful deterrent than expected, but additional themes included unexpected preconceptions toward games, social identity factors, and themes involving trust; students simply did not trust that a game with an educational purpose would be worth their time. A larger theme was utilitarian vs hedonic evaluation: once the game was introduced as ‘educational’, the students assessed it for its utility value, not just its hedonic value. Utilitarian assessment of educational games appears to be an underexplored factor in educational game adoption, particularly in adult users such as postsecondary students. These interviews, along with an examination of exergame usage as an analogue for adult educational game users, suggest that balancing utilitarian and hedonic mechanics is key. Further research is needed to identify an ideal balance for “fun tools” to enhance educational game adoption rates in post-secondary. iii Table of Contents Abstract ............................................................................................................................... ii Table of Contents ............................................................................................................... iii List of Tables ................................................................................................................ vii List of Figures .............................................................................................................. viii Dedication ...................................................................................................................... ix CHAPTER 1: Background .................................................................................................. 1 Utilitarian Evaluation of Educational Games in Post-Secondary Students .................... 1 Educational Games ......................................................................................................... 2 Educational Game Design........................................................................................... 3 Engagement................................................................................................................. 4 Hedonic Mechanics and Triangularity ........................................................................ 6 Current State of Post-Secondary Educational Games..................................................... 7 Research Questions ......................................................................................................... 9 CHAPTER 2: Study Design and Methods—Part One: Assessing Triangularity ............. 10 The ‘Playful Experience’ Framework and Questionnaire (PLEXQ) ............................ 10 Hypothesis .................................................................................................................... 11 Study Design and Game Creation ................................................................................. 11 Gameplay Design and Aesthetics ............................................................................. 14 Triangularity vs Non-Triangularity Versions ........................................................... 15 Sampling and Evaluation .......................................................................................... 16 Initial Trial and Amendments Regarding Reward for Participation ............................. 17 iv CHAPTER 3: Results—Part One: Game Interaction Data ............................................... 21 Initial Trial—Fall 2022 and Winter 2023 ..................................................................... 21 Fall 2022: XMAT Game Interactions ....................................................................... 21 Fall 2022: MATH 190 Game Interactions ................................................................ 22 Winter 2023: MATH 190 Game Interactions ........................................................... 22 Second Trial—Fall 2023 ............................................................................................... 23 Fall 2023: XMAT Game Interactions ....................................................................... 23 PLEXQ Results ............................................................................................................. 26 CHAPTER 4: Discussion—Part One: Assessing Triangularity ....................................... 27 Limitations of Part One................................................................................................. 27 Refocusing of Research Question ................................................................................. 30 CHAPTER 5: Study Design and Methods—Part Two: Assessing Adoption ................... 31 Interview Invitations ..................................................................................................... 31 Previous Participants................................................................................................. 31 Winter 2024 Interview Cohort .................................................................................. 31 CHAPTER 6: Results—Part Two: Themes from the Interviews ..................................... 33 Winter 2024: MATH 190 Game Interactions ............................................................... 33 Interview Data Collection ............................................................................................. 34 Themes From the Interviews ........................................................................................ 35 Triangularity vs Non-Triangularity........................................................................... 35 Enjoyment, Motivation, and Perception of Content ................................................. 35 Theme and Immersion in Narrative .......................................................................... 37 v Student Expectations and Preconceptions ................................................................ 39 Preconception of Game Design and Traditional Educational Games ................... 39 Perceptions of Games as a Hobby and Gaming Purpose ...................................... 41 Perceived Usefulness and Utilitarian vs Hedonic Perceptions ............................. 42 Avoidance of Math Content .................................................................................. 45 CHAPTER 7: Discussion—Part Two: Assessing Adoption............................................. 48 Reaction Themes from the Interviews .......................................................................... 48 Trust Reactions ......................................................................................................... 48 Complexity Reactions ............................................................................................... 50 Social Identity Reactions .......................................................................................... 51 Math Content Reactions............................................................................................ 51 Limitations of Part Two ................................................................................................ 53 CHAPTER 8: Utilitarian Evaluation of Educational Games ............................................ 54 Summary of Data Analyses: Hedonic and Utilitarian Values ...................................... 54 Overall Expectations and Assumptions ........................................................................ 56 Existing Literature: Exergaming as an Analogue ......................................................... 58 Critical Incident Theory ................................................................................................ 59 CHAPTER 9: Research Gaps and Future Research ......................................................... 61 Potential Themes to Investigate for Future Research ................................................... 61 Utilitarian Evaluation vs Hedonic Evaluation in Program Adoption ....................... 61 Snap Evaluation and Potential Branding Issues........................................................ 62 vi Perception of the Purpose of Games ......................................................................... 62 Reactions to Math Content........................................................................................ 63 Educational Games vs Simulators ............................................................................ 64 Resiliency in Gamers vs Non-gamers ....................................................................... 65 Narrative Enhancing Immersion ............................................................................... 66 Cognitive Load and Exam Preparation ..................................................................... 67 CHAPTER 10: Conclusion ............................................................................................... 68 References ......................................................................................................................... 70 Appendix A: Educational Game Design ........................................................................... 74 The LM-GM Model ...................................................................................................... 74 Examples of Educational Game Design in Use in Post-Secondary Education Today . 77 Oregon Trail (1985) .................................................................................................. 77 Potluck Panic (2016) ................................................................................................. 79 Malaria Invasion (2019) ............................................................................................ 82 Academic Integrity in Cyberspace (2023) ................................................................ 87 The State of Post-Secondary Educational Games......................................................... 91 Design of the Game in this Study ................................................................................. 92 vii List of Tables Table 1 Fall 2022 Player Usage Data for XMAT ...................................................... 21 Table 2 Winter 2023 Player Usage Data for MATH 190 ......................................... 23 Table 3 Fall 2023 Player Usage Data for XMAT 161 .............................................. 24 Table 4 Fall 2023 Player Usage Data for XMAT 162 .............................................. 25 Table 5 Winter 2024 Player Usage Data for MATH 190 ......................................... 33 Table 6 Comments related to Motivation or Perception of Content ......................... 35 Table 7 Comments Related to Expectations of Students .......................................... 39 Table 8 Comments Related to Perceptions of Games and Gaming .......................... 41 Table 9 Utilitarian vs Hedonic Comments ............................................................... 42 Table 10 Utilitarian vs Hedonic Comments by Player Category ............................... 43 Table 11 Utilitarian vs Hedonic Comments by Type ................................................. 43 Table 12 Utilitarian vs Hedonic Comments by Type – Math Comments Separated . 46 Table 13 Positive vs Negative Comments by Type – Math Comments Separated .... 47 viii List of Figures Figure 1 An Illustration of “Triangularity” (Schell, 2008) ............................................... 7 Figure 2 The LM-GM model combined with Bloom’s Taxonomy (Arnab et al., 2015) 74 Figure 3 A game map demonstrating the LM-GM model (Lim et al., 2015) ................ 75 ix Dedication I would like to dedicate this work to the memory of Dr. Lisa Poirier. Lisa pushed me to take on challenges that I would not otherwise have had the confidence to tackle, which led directly to the position I am in now as a staff member at UNBC. Lisa was one of the sponsors for my graduate application, and had expressed interest in testing some of my game designs in her courses. We never got the chance to do that. UNBC, and I, will miss her greatly. 1 CHAPTER 1: Background Utilitarian Evaluation of Educational Games in Post-Secondary Students I am a gamer. My internet friends, who are also gamers, call me a “capital-G” Gamer. I play many different genres: everything from Dark Souls to Power Wash Simulator. I remember rushing to the computer lab in elementary school, trying to claim the 386 with the colour monitor so that I could play Number Munchers, but if someone claimed it first, at least I could still play Oregon Trail in monochrome. We thought we were witnessing the future of education—we would have computers in our homes, we would play games on them, and we would learn. Gaming taps into base human reward mechanisms, such as “flow”, described by Csikszentmihalyi (2014) as being fully immersed in an activity, and “fiero”, which is an Italian word for “pride” or the “victory rush” that one experiences after a triumph (McGonigal, 2011). Because of those reward systems, gaming seems ideal for capturing increased cognitive effort and active attention in an attempt to reap increased gains in learning by doing. It is therefore unsurprising that educational games became a focus when computers became more commonplace in the home, potentially giving children access to educational content outside of the classroom in a format they might be excited to engage with. What is surprising is that educational games did not become the dominant genre that was predicted in the 1980s (Richtel, 2005). Currently, educational games are almost entirely relegated to elementary school ages, and are essentially non-existent at the post-secondary level aside from occasional projects that are developed and used in-house. There appears to be no current commercial market for educational games at the post-secondary level, and yet the feedback the Academic Success Centre at the University of Northern British Columbia (UNBC) has received from students when games are incorporated into post-secondary events, like orientation or study-focused activities, is generally 2 favourable (Academic Success Centre, personal communication). Post-secondary education appears to be an untapped market for educational game development. Educational Games An educational or “serious” game is any game that is designed for a purpose other than entertainment (Djaouti et al., 2011). Educational games have a storied history in the gaming industry, bursting onto the scene in the 1980s and appearing in schools across the world. One example of a successful educational game is the still-popular 1985 game of Oregon Trail (Classic Reload, n.d.; Gjarnjobst, 2017), which asked the player to lead their family along the real-world path of the pioneers of the 1800s, from Kansas to Oregon. In the process of playing, students were exposed to real-world history and learned of the perils the original pioneers faced in order to settle the west. The success of Oregon Trail and other industry pioneers led to more attempts to utilize gaming in educational contexts, and the future of computerized education seemed almost guaranteed to include gaming in some format. Despite the initial boom of the educational gaming industry, software giants like The Learning Company became defunct, and today’s educational games are primarily focused on elementary skills for younger children and are a relatively small part of the market (Richtel, 2005). Much of the available educational software focuses more on gamification (adding game elements in an attempt to increase persistence), such as with language learning software like DuoLingo (Huynh et al., 2016). Gamified training programs have their own appeal and functionality, but they seem to be missing the mark that made educational games like Oregon Trail so appealing. Players played (and sometimes still play) Oregon Trail because they were interested in the game itself, not just because they wanted to hit the next milestone or earn a new badge. In 2007, in an article on Slate.com, Justin Peters wrote “Ever since video games were 3 invented, parents and teachers have been trying to make them boring” (para. 1), and that seems to be indicative of the design problems facing many of the educational games that are available today: the focus of the game seems to be on the learning outcome as the primary objective rather than on the mechanics that will make the game “fun”. The proliferation of these “boring” educational games have burdened the market with a perception that educational games are more homework than they are entertainment. Educational Game Design Malone (1982) describes the difference between a toy, where a user makes their own goals for the use of the object, and a tool, where the object is created with a specific goal in mind from the start. Many educational games seem to fall into the latter category—they are solely designed to teach a specific subset of content to a student. The result is more of a tool that emphasizes the utilitarian aspects over the hedonic ones—an ‘interactive lecture’ or flashcards— than a toy that the user can immerse themselves in. McGonigal (2011) emphasizes the role of player choice in gaming. If the player does not choose to play the game, then it is no longer a game—it is a mandatory task. For example, a quiz show or trivia contest might be fun for a player who chooses to challenge themselves, but a quiz that is required to pass a course is generally considered more stressful than fun. Educational games may violate this design principle by making it mandatory to play. A major goal of the educational games industry should be to incorporate a careful balance of utilitarian and hedonic aspects into game design, ensuring that educational games are games, not just tools. Educational games should be designed so that the player feels that they have agency, and therefore volunteers to interact with the content within the gaming wrapper. The player needs to feel free to immerse themselves and discover a hedonic experience. If the player 4 is evaluating the game purely on its usefulness or its learning outcome, they are using it as a tool, not as a game or toy. Engagement Engagement is a necessary first element for player persistence. A player may begin playing a game for any number of reasons (including being told to try it), but if there is no engagement, it is unlikely that they will persist in playing it. Much like a student who is no longer directing attention to their study material, no further learning advancement can occur if a player is not engaged in a game. Schoenau-Fog (2011) describes game “engagement” as “… when players dedicated themselves to coming back and playing a game again and again” (p. 4), as well as a “process whereby players engage in a pursuit of objectives and consequently perform a range of activities in order to accomplish objectives and feel affect” (p. 16). Engagement does not automatically indicate enjoyment. The affect experienced by the player may not necessarily be positive—they may not be ‘enjoying’ their experience with the game—but they may still feel compelled to return to it, indicating engagement (Schoenau-Fog, 2011). Scott Osterweil (as quoted by Jenkins, 2006) said “the fun of gameplay is not non-stop mirth but rather the fun of engaging of attention that demands a lot of you and rewards that effort” (para. 6). Jenkins (2006) noted that a lot of gameplay elements are not necessarily fun as you perform them: ‘grinding’ in a video game, where the player performs repetitive tasks to master a skill or obtain the resources necessary to accomplish a future goal, can be much like completing homework or doing practice problems while studying. McGonigal (2011) relates games to “satisfying work”—a system that gives the player work to do with a clear goal, provides feedback for them about that work, and rewards them for a job well done. It may seem counterintuitive to consider games to be ‘work’, but the analogy 5 holds when considered against gaming mechanics that should not seem compelling, like ‘grinding’. The key is for the player to have a clear and salient goal, a ruleset that makes that goal a reasonable challenge, and feedback to show the amount of progress they have made toward the goal (McGonigal, 2011). Completing the tasks has the potential to put the player into a state of meditative flow, and reaching the goal has the potential to give them the addictive rush of ‘fiero’, which then keeps the player engaged and persistent. The most important aspect is that the goal—the point where they will achieve the hit of ‘fiero’—is something that matters to the player (and not necessarily the educator), and thus they remain engaged and motivated (McGonigal, 2011). Ryan et al. (2006) describe gaming motivation in the context of self-determination theory, particularly around the concept of “autonomy” when gaming. When someone is performing an activity for personal interest, their autonomy is high, and they are likely to be intrinsically motivated to persist in that activity. If the activity is a mandatory task, their autonomy is likely not as high as if they had decided to do it for personal interest, and they will be less motivated to engage in the task. Whether mandatory or self-directed, if the activity contains events or conditions that remove control from the person it can impact their autonomy, and therefore their motivation, and result in a tendency to disengage (Ryan et al., 2006). Ryan et al. (2006) also discuss a need for “competence”, described in the terms of challenge and effectance. According to Ryan et al., games provide the opportunity to acquire new skills, but the player needs to be “optimally challenged”, which aligns with the Zone of Proximal Development (Vygotsky, 1978). Having a safe platform to practice skills allows the player to experiment and feel accomplishment with the material, boosting satisfaction and providing a feeling of competence that leads to intrinsic motivation (Ryan et al., 2006). 6 Schoenau-Fog et al. (2011) indicate that users disengage when there is nothing at stake, but Arrasvuori et al. (2011) indicate that users disengage when there is too much at stake and stress is too high. Both of these stances on stakes may be correct: Malone (1982) said that “for an activity to be challenging, it needs to have a goal whose outcome is uncertain” (p. 65), and further described how the outcome “becomes certain” for activities that are too easy (a guaranteed win) or too difficult (a guaranteed loss). The design goal, therefore, should be a middle ground where the stakes are salient and relatable, generating user engagement, but still ‘safe’—enabling experimentation and learning. Hedonic Mechanics and Triangularity The majority of research into educational game design focuses on how to increase the hedonic aspects and experience of games, such as immersion and engagement (Hamari et al., 2016), and hedonic aspects have been heavily considered in models related to educational game design, such as the Hedonic-Motivation System Adoption Model (Lowry et al., 2013). Despite the focus on hedonic aspects, very little research has categorized specific game mechanics or prioritized them for their effect on the hedonic perception of a game. Schell (2008) documented a number of game mechanics, which he styled as “lenses” to view the game through. One lens was that of “triangularity”, named as such because it (roughly) forms a triangle (Figure 1). For a game to offer true triangularity, the player must be faced with choices. In one choice, the player will assume a low amount of risk (down to and including a lack of risk), and will progress at a slow and steady rate. The other choice should be a much higher risk, possibly with the potential to set the player back in terms of game goals, but with much greater reward. 7 Figure 1 An Illustration of “Triangularity” (Schell, 2008) Note: From “The Art of Game Design: A Book of Lenses,” by J. Schell, 2008, Elsevier (http://doi.org/10.1201/9780080919171). Copyright 2008 by Elsevier Inc. Used with permission. Triangularity is a hedonic mechanic that keeps the player engaged by forcing them to make interesting decisions. The allure of the greater reward must be balanced by the potential risk of losing progress, otherwise there would be no point in ever choosing the low-risk option. However, the low-risk option should still provide the player with an avenue to advance without becoming frustrated and desiring to quit, losing engagement. Current State of Post-Secondary Educational Games To begin this study, a review was conducted to look at the current state of post-secondary educational game design, but almost all current literature focuses on design for elementary school ages which is not an equitable comparison. With triangularity identified as a mechanic of interest, the review then turned to searching for examples of educational games that were developed for and used in post-secondary environments, with a focus on determining whether triangularity was incorporated into the design. There was a limited selection of still-accessible post-secondary games available to review, with many of them having been abandoned or depreciated, particularly after “Adobe Flash Player”, which many of these games had previously been developed for, was discontinued in January 2021. A subsequent review of the games that 8 were still accessible revealed that very few of the current post-secondary educational games seem to use triangularity as a design mechanic (Appendix A). It is unclear if triangularity is neglected because it is unknown, or if it is avoided because educators have a focus on learning outcomes and choose not to add risky choices that could potentially confuse or discourage the player. Yet, the risk vs. reward choices provided by triangularity seem to play a pivotal role in boosting the hedonic experience of a game—making a game interesting and engaging to a player. Triangularity could be seen as simulating the reward schedules seen in gambling or in a “Skinner box” phenomenon (Skinner, 1953/2014), where an action results in a reward, but a different action may result in a bigger reward—reinforcing the player’s actions, providing satisfaction from ‘winning’, and resulting in persistence (sometimes to a pathological degree). There is concern about the Skinner box mechanism moving the participant’s motivation from intrinsic to extrinsic (Deci, 1971), and thereby harming persistence, but that can be addressed to some extent if the triangularity reward is progress itself and is inherent to proceeding through the narrative of the game. The frustration of reward schedules is mitigated by the presence of the low-risk/low-reward option where the player makes an active choice to receive a smaller reward, with the knowledge that they could have received more if they assumed more risk. It is my belief that player choice needs to be a fundamental hedonic game mechanic, and those choices need to hold meaning for the player. In order for a choice to hold meaning, there must be consequences associated with the choice—an introduction of ‘stakes’, yet still within the category of ‘safe’ or ‘playful’ stakes. Instilling triangularity into the design of a game helps fulfill the need for meaningful player choices. 9 Research Questions This study set out to investigate whether an educational game targeting a post-secondary audience would be more preferred by students if it involved triangularity, compared to a ‘traditionally-designed’ educational game. The intention of this investigation was to make an argument that triangularity needs to be emphasized as a necessary game design element in order to improve educational game design and bring new life to the genre, as well as highlighting postsecondary education as a potentially under-utilized market for game development. The initial trials of the study revealed an unanticipated result: participation was so low that it made it impossible to gather any meaningful data. The assumption upon commencement of this study was that presenting the content in the framework of a game should have been entirely beneficial, with no downsides, and might spark curiosity and encourage a student to explore the course content more than they might have with traditional study methods. In reality, the students chose traditional study methods over the game. It was not that the students explored the game and then decided to continue with traditional methods—the students actively avoided the game. The design of the game, triangularity or not, had no bearing on whether the students played, because they chose not to engage with it in the first place. In response to this revelation, the study gained a second research question: why would students choose traditional study methods over a ‘study game’, marketed as ‘fun’, that should have fulfilled the same goal? What factors prompted outright rejection of the game as a method, without any curiosity or exploration from the students? As a result, this study was conducted in two parts: the first part presented the game to students and observed their interactions with it, and the second part investigated their personal reasons for engaging or not engaging with the game. 10 CHAPTER 2: Study Design and Methods—Part One: Assessing Triangularity Having identified triangularity as an often-neglected element in educational games, and recognizing that designing a game with triangularity will increase the complexity of development, it seemed prudent to assess whether triangularity improves game engagement as compared to a design that excludes it. The Playful Experiences Questionnaire (PLEXQ) tool (Arrasvuori et al., 2011; Boberg et al., 2015) provides us with the opportunity to gauge user experience. The ‘Playful Experience’ Framework and Questionnaire (PLEXQ) Arrasvuori et al. (2011) distilled the user experience into serious and playful states, essentially defining a serious state as doing a task for real and permanent consequences (such as when writing a final exam) as compared to a playful state where the outcome has lower stakes and the user is free to experiment and learn in a ‘safe’ environment (such as a simulation, or mock exams). According to Arrasvuori et al. (2011), a user in a playful state feels safe to put effort into doing a task and is less likely to feel overwhelmed by the task even if it is significantly challenging. This ‘safe’ playful state makes it more likely that the user will engage with the task, making it an ideal first step for learning the skills necessary to complete the serious task later. To this end, Arrasvuori et al. designed the “Playful Experiences (PLEX)” framework to assess whether a user is experiencing a playful state or a more serious non-playful state when interacting with a game, and to help highlight consideration of these elements in game design. The PLEX framework has been tested as reliable and valid, and has further been refined into the “PLEXQ” questionnaire which can be used as an exit survey to assess user experience (Boberg et al., 2015). 11 In order to assess triangularity in educational games, two versions of a game need to be created: one with triangularity and one with a more traditional non-triangularity educational game design. Players need to be randomly assigned to a version, spend some time playing it, and then respond to the PLEXQ with their assessment of the game. The PLEXQ uses Likert scales, which means it is possible to do a statistical analysis on the average scores to compare the player experience between two versions of a game. Hypothesis The hypothesis is that an educational game that includes triangularity in its game design will be more positively received by students, as measured by their usage of the game and rating on the PLEXQ, than an educational game that does not include triangularity in its design. The null hypothesis is that there would be no difference in student reception between the two versions. Study Design and Game Creation The instructor for “Math for Elementary Teachers” (MATH 190) and “Intermediate Algebra” (XMAT) at the University of Northern British Columbia (UNBC) agreed to allow a game to be developed using the course materials and administrated to students in these courses. These courses were considered to be ideal candidates for an educational game because they feature content that requires repeated practice (algebra and pre-calculus content) and are requirements for non-mathematical degrees, meaning the students in these courses are typically not inclined to seek out mathematical content unless it is required. Since an educational game is intended to make unpalatable content more palatable, it is ideal to find students who are studying something they are less passionate about. The MATH 190 course was additionally considered ideal because much of the course content explores using math in games and making it fun for 12 elementary children to explore, so it was expected that there would be interest from these students even if only to explore the concept of an educational game as a delivery method. Mathematics courses were not specifically targeted other than through convenience of having an instructor to partner with—this game could have been designed for a non-mathematics course, but there was an existing partnership with a mathematics instructor and instruction effects were an unknown factor that would have to be controlled for, so no additional nonmathematics courses were targeted. The instructor uses a free open-source textbook (OpenStax, 2023) and provides the students with a comprehensive list of ‘goal questions’ that are handselected from the practice problems in that textbook. Having instructor-endorsed study material provides the perfect base of information to create game questions from, because it means the game designer does not need to be an expert on the material to be able to construct the game content. Students in these courses would already have access to the textbook and goal questions, so presenting the content in a game merely represents a different way of accessing the study material, and not an inherent advantage or difference in the content that is covered. A physical (cardboard) version of the game was considered, with the goal of having students play the game in a monitored setting to get their reactions in real-time. However, administering the game during class time would impact the player experience, both in that their classmates would be watching them play (and attempting to solve math problems, adding additional stress and performance anxiety), and that the spectators would see the game being played which would change their experience with it when it became their turn. Even though it increased the complexity of development, it was decided that there must be a digital version of the game that players could access at any time and play with at their own pace without observers. Ideally, a digital game would also track whether the user returned for more than one play session 13 and the total time each user spent playing, providing additional data points on engagement that would not be present in a structured and monitored play session. Using a digital interface introduces other design concerns, such as how an unintuitive interface might affect the user’s experience; however, the potential benefits offset the extra complexity of development. At UNBC, students in these courses have access to Moodle, a Learning Management System (LMS), where course information and materials are delivered. Articulate Storyline 360 (Articulate Global, 2023) is a software that is specialized for developing interactive course content that can be delivered through an LMS, and while it is not designed to be a game engine, the software has access to elements such as variables and triggers which allow it to be used to develop game-like mechanics which can then be delivered through an LMS such as Moodle. The LMS allows the players to access the game through a browser interface, which means no additional software installation is required, nor are any extraneous accounts required to be created beyond their login credentials for the course LMS itself, which helps reduce barriers of access for the player. The Articulate Storyline 360 software is designed to be accessible to nonprogrammers, which means an educator could reasonably expect to be able to develop a game experience for their course with this software regardless of programming experience, making it an ideal development environment for this investigation. The game content was crafted from the goal questions that were provided by the instructor, with the wording and arrangement of the problems copied directly from the OpenStax textbook to preserve the format and presentation of the content. Graphics for the game were pulled from the library of images provided within Articulate 360, which are free to use as part of the license agreement for the software. 14 Gameplay Design and Aesthetics The game was given a sci-fi theme and set on a spaceship. The player was greeted with a welcome message that explained that their spaceship was damaged and that they need to repair the ship’s engine before oxygen runs out. Sci-fi was chosen as a theme because it felt intuitive to need to do mathematics when repairing a spaceship, but the overall theme is a small part of the game and could easily be swapped with graphics fitting another theme instead. Gameplay consisted of selecting different “subsystems” to prepare to repair the engine. Each “subsystem” corresponded to a chapter in the course and contained the goal questions selected by the instructor for that chapter. The “engine” was a series of questions pulled from a final exam provided by the instructor and used with permission. To repair the engine, the player had to answer all of the questions in a timed sequence, similar to taking a timed exam. The instructor of the course could complete the engine question series in eight minutes, so the player was provided with a default ten minutes, meaning someone competent with the source material could go straight to the engine question and complete it without doing any other activities in the game; however, it was expected that the average student in these courses would need much more than ten minutes to complete the sequence. The engine was the only part of the game that had an active timer. The clock was visible, so that the player could view how much time they had accumulated, but the clock did not tick down when the player was working on the “subsystem” questions. Each correctly-answered subsystem question provided more time for the clock, and there was no limit, allowing the player to collect as much time as they needed to feel comfortable before attempting the engine repair sequence. In addition to accumulating time on the clock, the game tracked how many questions had been correctly answered in each subsystem, with a status screen that showed a bar beside each 15 chapter that visibly filled, so that the player could keep track of how many questions they had answered for each chapter. Additionally, the game board showed a checkmark for chapters that the player had successfully completed, and the checkmark changed colours and grew brighter if they returned to the chapter to do more work. These elements gave the player a visual prompt that their actions were having an effect on the game state, and allowed them to keep track of which chapters they spent the most time on. Articulate 360 allows the player to save their game state and return to it, so there was no need to complete all chapters in a single session, and the player could return to the game to tackle new chapters as they progressed through the course content. They could choose to reset their game progress at any time, and if they successfully repaired the engine and completed the game, they could choose to re-enter the module and start again for more practice. Triangularity vs Non-Triangularity Versions In the non-triangularity version of the game, the player could repair subsystems to accumulate time on the engine clock, but an incorrect response to a question merely resulted in no time accumulation. There was no penalty for an incorrect response, and there were no decisions to be made apart from which chapter the player wanted to tackle in the game session. The triangularity version of the game added an additional layer of choice where the player must decide if they wanted to attempt a subset of less-complicated questions from the chapter they selected, or if they would like to attempt a more-complicated and potentially moredifficult question, but receive a higher reward for doing so. The instructor of the course assisted with categorizing the goal questions into more- and less-complicated. If the player chose the less-complicated path, they received less-complicated questions, typically questions from earlier in the chapter or the multiple-choice style questions where it is 16 easier to eliminate wrong answers without having to calculate the entire question, and received an engine time bonus for a correct answer. Incorrect answers incurred no penalty, and the player was free to attempt each subsystem as many times as they wanted. The less-complicated path had a limit on it, so the player could receive no more than five rewards from correctly solving the less-complicated questions. Once the player reached the attempt limit, they received a message informing them that they could earn no more engine time from this path but that they were welcome to continue to play it for practice if they wanted. Further attempts would not reward time bonuses, but the attempts would still be counted on the status screen. If the player chose the more-complicated path, they received questions from later in the chapter and questions that required more calculations with fewer hints available. A correct response on these questions rewarded more time than the less-complicated path, typically three minutes instead of one minute, however, an incorrect response subtracted time from the clock, adding an additional penalty that was not present in the less-complicated path. The morecomplicated path was unrestricted, and the player could continue to accumulate (or lose) time with unlimited attempts. In this way the more-complicated path added triangularity, where the player must choose between the simpler but more limited path, where they could potentially accumulate five minutes of time per chapter with no risk, or the more lucrative, more complicated path where they could accumulate unlimited time at a rapid pace, but risk losing some of their accumulated engine time if they were less confident with the material for a chapter. Sampling and Evaluation To reduce effects brought about by students communicating between groups, it seemed best to offer the non-triangularity version of the game to a cohort in one semester, and offer the triangularity version of the game to a new cohort in the same course in a separate semester 17 (being taught by the same instructor, to minimize variables between semesters). All students within the course were allowed to self-select whether to play the game or not. The game content was built from the textbook and goal questions provided by the instructor, so all students in the course had access to the same material. The only difference should have been the presentation of the material: in a traditional study setting (textbook or supplemental materials), in a nontriangularity game, or in a triangularity game. Once the students participated in the game, they were asked to complete the PLEXQ questionnaire (Boberg et al., 2015) in order to evaluate their level of engagement and experiences of playfulness in the game. The dates of all exams were noted in a calendar and reminders were sent in Moodle one week prior to every major exam with instructions on how to access the game. The reminders included encouragement on how the game questions are also study materials to help prepare for the upcoming exam, and instructions on how to fill out the PLEXQ survey once they had played with the game. All students had the opportunity to opt-out of the study at any time, and opted-out students would not be contacted for surveys or other forms of follow-up. The opportunity to debrief was provided for any student who became upset or distressed during the study. Initial Trial and Amendments Regarding Reward for Participation The initial trial of the study was run in Fall 2022 and Winter 2023, allowing students in both XMAT and MATH 190 to self-enroll in the study and utilize the game unrestricted. The Research Ethics Board had suggested attaching an incentive to participation, but there was a desire to examine whether the game would be intrinsically motivating and encourage students to study (by virtue of playing the game, and thereby interacting with the goal questions that were marked as study materials by the instructor). As per Ryan et al. (2006), making the game a 18 mandatory task affects the autonomy of the players and impacts their experience with the game, and attaching a reward might make participation feel more ‘mandatory’, so there were no rewards attached to participation in order to explore whether the triangularity version resulted in more intrinsically-motivated studying than the non-triangularity version. The research project was presented to the classes in-person in the first week to introduce the students to the study, the process of consenting to access the game, and a description of the game itself and how it works. This presentation emphasized that studying and practice was necessary in order to succeed in the course, and that the game was built using the goal questions provided by the instructor, and therefore represents instructor-endorsed study material. The game was made available in the course shells in Moodle, and any student could gain access by responding affirmatively to the consent “quiz”, which acted as a gate to unlock access to the game. The game was available for the entire semester, and Moodle announcement reminders about the game and the process of unlocking it were sent one week before every exam. Unfortunately, there was very low participation from both courses in the first trials, and it was determined that following the suggestions from the Research Ethics Board to attach a reward might encourage participation in the study. The Research Ethics Board approved a reward of a 1% bonus mark (set and controlled by the instructor) for participation in the trial for the Fall 2023 and Winter 2024 semesters. Students who did not want to participate in the study were able to opt to submit the goal questions to the instructor as an assignment to receive the bonus mark, so that every student in the course had the same opportunity to earn the bonus. To be eligible for the bonus mark, students would have to consent to the study, answer at least one question correctly in the game, and then respond to the PLEXQ survey, which would not unlock in Moodle until the student had interacted with the game, to prevent students from skipping to the 19 survey to earn the bonus mark without participating. The instructor of the course offered to help any student ensure they got at least one answer correct if they were concerned about being excluded from this reward. Due to time constraints, it was also necessary to pivot from the preferred between-groups design, which compares one semester of students to another, to a within-groups design which allows the same student to try both versions of the game and fill out the survey twice, outlining their experiences with each version. The change to design meant that the results are subject to practice effects; therefore, the order in which the student experienced the games was randomized. The study design was also subject to effects from the students communicating with each other about the game, which may have influenced their experience if they heard about another student’s experience with a different version before they tried it. Even though it was somewhat problematic, running the study as a within-groups design allowed data collection to complete within a single year, whereas running the study with the between-groups design would require another full year to run the second version of the game in the same course environment. There was no guarantee that the course would be run by the same instructor in a new year, which would cause additional effects or possibly the loss of faculty participation for that particular course, which would be catastrophic for data collection. In the second trial, during the Fall 2023 and Winter 2024 semesters, consenting students were randomly sorted into two groups which determined which version of the game they would receive. Group One was given access to the non-triangularity version in XMAT 161 (spanning the month of September), and then given access to the triangularity version for XMAT 162 (spanning the month of October), while Group Two received the triangularity version for XMAT 161, and the non-triangularity version for XMAT 162. Sorting the groups in this way would 20 allow for comparison of their comments to the two different versions as a within-groups trial. Both versions of the game were then made available for XMAT 163 (spanning the month of November, and completing the semester), to see whether students returned to the game when there were no more bonus marks attached, potentially indicating intrinsic motivation, and which version of the game they used more, if any, which might reveal which design was preferred. In the first week, the same presentation and script were used to introduce students to the game, with added directions to explain how to attain the bonus mark. The same reminder schedule was used as before, sending an announcement via the course shell with links leading directly to the consent form and game, while emphasizing that the game provided the goal questions that were recommended study material for the upcoming exams, and that full participation in the study would accrue a 1% bonus mark award. 21 CHAPTER 3: Results—Part One: Game Interaction Data Initial Trial—Fall 2022 and Winter 2023 The game was provided to students taking “Intermediate Algebra” (XMAT) and “Math for Elementary Teachers” (MATH 190) at the University of Northern British Columbia (UNBC). XMAT took place on campus for Fall 2022, while MATH 190 was held online. In the Winter 2023 semester the game was provided to students taking MATH 190 on campus. Fall 2022: XMAT Game Interactions Out of 23 students enrolled in XMAT in Fall 2022, eight consented to the study. Only seven students accessed the game. One of the seven students accessed the game twice, but each time closed it before attempting a question. Two of the students attempted two and three questions each, logging a total of three minutes and five minutes total playtime respectively. The remaining four students answered 15 questions for a total playtime of 23 minutes, 3 questions for a total playtime of 25 minutes, 19 questions for a total playtime of 70 minutes, and 79 questions for a total playtime of 424 minutes (seven hours) respectively (Table 1). Table 1 Fall 2022 Player Usage Data for XMAT 3 # of Questions Attempted 15 Time Spent Playing (minutes) 23 2 2 3 5 3 2 2 3 4 1 3 25 5 10 79 424 6 2 19 70 7 2 0 0 Student Number of Sessions 1 22 Two students from the Fall 2022 XMAT group responded to the PLEXQ survey. A third student began the survey but only completed the demographics section, so their response was discarded. The two students who responded to the survey indicated that their favourite thing about the game was how the added narrative made solving the problems more compelling, and how the game provided a variety of different ways of interacting with the material. Both suggested that the game should have more robust feedback when a wrong answer is submitted. Fall 2022: MATH 190 Game Interactions Of the 17 students enrolled in MATH 190, four consented to the study. Only one of those students launched the game, but they closed the game without attempting a question. No students in MATH 190 attempted any questions in the game or accumulated any playtime over the semester. Subsequently, no students from MATH 190 responded to the PLEXQ survey. Winter 2023: MATH 190 Game Interactions Of the 18 students enrolled in MATH 190, 12 consented to the study. Of the 12 students who consented to the study, eight accessed the game. Two of the students who accessed the game only had one session each, attempting one question each for a total playtime of one and five minutes respectively, and never returned to the game. Three of the students had two sessions but spent fewer than two minutes per session attempting two and three questions respectively, for a total playtime of fewer than five minutes each. Only two of the students spent more than 10 minutes with the game, with one attempting nine questions in a single session with a total playtime of 16 minutes, after which they did not return to the game, and one attempting 15 questions over four play sessions, for a total playtime of 36 minutes (Table 2). 23 Table 2 Winter 2023 Player Usage Data for MATH 190 3 # of Questions Attempted 3 Time Spent Playing (Minutes) 7 2 1 1 5 3 1 1 1 4 2 2 5 5 4 15 36 6 1 9 16 7 2 3 4 8 2 0 2 Student Number of Sessions 1 No students from MATH 190 in Winter 2023 responded to the PLEXQ survey. Second Trial—Fall 2023 Fall 2023: XMAT Game Interactions Of the 26 students enrolled in XMAT for Fall 2023, 14 consented to the study. The participants were divided into two groups consisting of seven students each, divided evenly between the non-triangularity and triangularity versions of the XMAT 161 game. Of the seven consenting students who were placed in the non-triangularity group, six launched the game. Of those six, two did not attempt any questions. One of the remaining four students had two sessions during the semester where they launched the game but did not attempt any questions, and a final session that occurred in the hour before the final exam where they launched the game and attempted seven questions for a total playtime of 41 minutes. The remaining students in the non-triangularity group had four sessions with the game for a total 24 playtime of 53 minutes, two sessions for a total playtime of 55 minutes, and one session for a total playtime of 76 minutes, respectively (Table 3). Table 3 Fall 2023 Player Usage Data for XMAT 161 Group One - Non-Triangularity Version Student Number of Sessions # of Questions Attempted 1 4 8 Time Spent Playing (minutes) 53 2 3 7 41 3 2 2 55 4 1 0 0 5 1 11 76 6 1 0 0 Student Number of Sessions # of Questions Attempted 1 1 20 Time Spent Playing (minutes) 81 2 2 2 9 3 1 3 3 4 3 9 14 5 2 7 29 6 1 7 18 7 2 2 4 Group Two - Triangularity Version All seven of the consenting students who were placed in the triangularity group launched the game. Of those seven, all students submitted answers to multiple questions in their play sessions. The triangularity group of students had one student who attempted two questions over two sessions for a total playtime of nine minutes, one student who attempted three questions in one session for a playtime of three minutes, one student who attempted nine questions over three sessions for a playtime of 14 minutes, one student who attempted six questions in one session for 25 a playtime of 28 minutes, one student who attempted seven questions in one session for a playtime of 18 minutes, one student who attempted 20 questions in four sessions for a playtime of 81 minutes, and one student who attempted two questions in a session right before the final exam for a playtime of four minutes, who then returned and launched the game an additional time the day after the final exam but did not attempt any further questions (Table 3). Students who progressed to XMAT 162, the second module of the XMAT course, were given access to the version opposite of the version they had for XMAT 161 (Table 4). This group of students was offered an additional 1% bonus mark for filling out the PLEXQ, with the intent of comparing the PLEXQ scores for the triangularity version against the non-triangularity version as a within trial. Table 4 Fall 2023 Player Usage Data for XMAT 162 Group Two - Non-Triangularity Version Student Number of Sessions # of Questions Attempted 1 1 10 Time Spent Playing (minutes) 32 4 1 2 33 6 1 1 1 5 1 5 7 Student Number of Sessions # of Questions Attempted 1 1 1 Group One- Triangularity Version Time Spent Playing (minutes) 1 In the XMAT 162 non-triangularity group (who had experienced the triangularity version in XMAT 161), four students returned to the game for one session each, spending 32 minutes, 33 minutes, 1 minute, and 7 minutes of playtime respectively (Table 4). In the triangularity group 26 (who had experienced the non-triangularity version in XMAT 161), only one student launched the game during the semester, and attempted only one question in it, for a total playtime of one minute (Table 4). Of the 12 students who played the game, only five responded to the PLEXQ survey to earn the bonus mark, with one of those five responding again in XMAT 162 for a second bonus mark. No other participants responded to the PLEXQ or attempted to earn a bonus mark in XMAT 162. One of the respondents submitted a response of “7” for every item, including conflicting items designed to catch invalid responses, which resulted in their response being discarded. The responses to the qualitative questions suggested that the students enjoyed the unique perspective of the added narrative, and one specifically highlighted that they enjoyed being able to choose the more difficult path as a challenge while still having the easier path to fall back on. Other comments were focused on game mechanics such as appreciating that they could resume after experiencing a connectivity outage, or discussions around how much time should be awarded, as well as a request for animations instead of static graphics. None of the students who responded to the PLEXQ returned to play the game after fulfilling the requirements for the bonus mark, even when there was ample time before the final exam to utilize the game for more study time. PLEXQ Results At this point, 84 students had been exposed to the game, but only seven had responded to the PLEXQ, with one of those responses being discarded as invalid. Ultimately, there were not enough responses to the PLEXQ to analyze or say anything meaningful about the results. 27 CHAPTER 4: Discussion—Part One: Assessing Triangularity The initial intent of this study was to investigate whether triangularity (a hedonic mechanic) had an impact on whether a student would choose to play an educational game. The ultimate result of that investigation was that the mechanics had no impact, because, even with an extrinsic reward, the majority of the students were not interacting with the game in any capacity. Additionally, Schoenau-Fog (2011) describes game “engagement” as “… when players dedicated themselves to coming back and playing a game again and again” (p. 4). Even when a student interacted with the game it was rare for them to return to it, with only three out of eightyfour participants across all cohorts exceeding an hour of play over the course of a semester. Very few of these interactions could be classified as “engaged” by the definition provided by Schoenau-Fog. A few preliminary assumptions could be made, such as how the students who experienced the triangularity version seemed more interested in trying other versions of the game, while those that tried the non-triangularity version had no interest in other versions, but no meaningful conclusions can be made from this amount of data. Limitations of Part One There are a number of limitations that may have affected the first part of this study. The study changed from a between methods design to a within methods design, which means some of the later sampling may be subject to issues like practice effects or be influenced by participants speaking to their classmates about a different version of the game. It was therefore inconsistent to make a direct comparison between early cohorts and later cohorts. The study design changed partway through part one to offer a reward, making direct comparisons between the early “no reward” cohorts and the “reward” cohorts inconsistent. However, since neither group engaged meaningfully with the game, this effect seems negligible. 28 Although the game was presented to several different cohorts of students, only a small number of students actually participated, which means there could be sample size effects. Since there were not enough responses to the PLEXQ to be meaningful, no analysis occurred, and because low participation was consistent between courses, the small sample size itself became a notable result. That said, the game had only been presented to four courses, 84 students total, at this point. The courses themselves were also small, with the largest containing 26 students. A small course enrollment could lead to greater effects of student communication or effects of not wishing their classmates to know they have been playing the game, therefore avoiding it. The courses chosen for this study, MATH 190: Math for Elementary Teachers, and XMAT: Intermediate Algebra, were chosen because the mathematical content requires repeated practice, and the students in these courses are not in a mathematics major and are therefore ideal candidates for an educational game designed to encourage interaction with material that they might be reluctant to interact with otherwise. There were also elements of convenience in that these courses were taught by the same instructor, reducing potential instruction effects, and that this instructor had pre-formed study material to hand out to the class, providing a convenient source of objective material and answers that are not affected by subjective interpretation. It is a limitation that the game was only offered in math-based courses, as math anxiety and efficacy are factors that can impact a student’s response to the game and its content, and there were no participants from non-mathematics courses to compare those responses against. The same instructor taught both courses, but the content of the courses differ so there could still have been instruction effects that changed how the students perceived the content or the game, making comparisons between cohorts inconsistent. 29 Even though the game was introduced to the classes with a presentation and explanation, including a description of how it might be a “fun way to study” and how studying is necessary to succeed in these courses, no demonstration of gameplay was offered to the students, meaning that their preconceptions and previous experiences with educational games could have shaped their expectations. Even though this game was presented as “fun”, exploring it to see what it was and how it functioned may have been perceived as a barrier, especially in the context of a mathematics course where the student may already feel anxious or overwhelmed. A demonstration of the game would have given the students a better idea of what to expect and may have potentially lowered the barrier to adoption by reducing uncertainty. The MATH 190 course in Fall of 2022 was offered entirely online, with no on-campus lectures. Even though the game was presented to the group during a synchronous lecture held over video conference, and was accessible in the course LMS in exactly the same way as it was for a student attending a course on campus, participation from this cohort was much lower than the other courses, suggesting that the online medium might have had an effect on student engagement with course material. All of the other cohorts in the study were meeting for lectures on campus, and are thus not directly comparable. Requiring students to do a survey represented an additional time commitment outside of time spent in the game itself. Even those students who tried the game and attempted to use it for their own studying would have to exert extra effort to respond to the survey, and knowing that the survey was a required part of the study may have discouraged students from attempting the game at all, creating a perception that the game was a larger commitment than it was. Offering a reward changed the dynamic of intrinsic interactions with the game, which was why the initial trial offered no reward. However, low participation resulted in the 30 investigation into intrinsic motivation being abandoned and a grade reward offered so that an investigation into triangularity could occur. Even with the reward, participation was very low, and some of the interactions were likely disingenuous attempts to accrue the reward with little effort. As a result, there was not enough genuine interaction to say anything meaningful about the effect of triangularity. Refocusing of Research Question From the perspective of the researcher and game designer, this study was providing a studying alternative that had no drawbacks: it was the exact same study material, presented in a novel format that might be more interesting than traditional study methods like doing practice problems repeatedly, without adding any complication to the practice itself because the questions were unchanged from the textbook. However, the students appeared to choose traditional study methods over the game, many doing so without investigating the game at all, which suggests that the design of the game had little bearing on their preference. The mere presence of an educational game, even with frequent reminders and endorsements from the course instructor, was not enough to motivate students to explore the game, and even a small extrinsic motivator of a grade reward was not enough for students to explore or adopt the game. In light of these results, the study was refocused to investigate adoption and explore why students were actively avoidant of an educational game. It was determined that it would be necessary to shift to a qualitative investigation and conduct in-depth interviews with the students who were exposed to the game in order to look for trends in the expectations and experiences surrounding the game. 31 CHAPTER 5: Study Design and Methods—Part Two: Assessing Adoption The first part of this study established that post-secondary students were reluctant to engage with an educational game that featured study material from their course, even when the game was presented in a positive light, endorsed by the course instructor, and incentivized with a small grade reward. In order to explore this reluctance, in-depth interviews had to be conducted with students who both did and did not choose to try the educational game. Interview Invitations Previous Participants The Research Ethics Board approved an amendment to conduct interviews. Invitations were sent to students who had previously consented to the study in both the XMAT and MATH 190 cohorts, offering a $10 gift card to a coffee shop at UNBC as compensation for a short interview. As part of this approval, the instructor decided to offer a 3% bonus mark as an incentive for students who were in-progress. Invitations were then sent to the Fall 2023 XMAT cohort, offering a choice of the gift card or a bonus mark. No students responded to the request. Seven students responded to the invitation to interview: five responses from the Winter 2023 MATH 190 cohort, and two responses from the Fall 2022 XMAT cohort. One of the MATH 190 responses declined the interview, but the other four agreed to interview. This first round of invitations resulted in interviews with six students: three who had played the game, and three who had not. Winter 2024 Interview Cohort In Winter 2024, the game was presented to the MATH 190 cohort, consisting of 33 students. The PLEXQ was dropped from the study at this point, as it would be unlikely to get enough responses for a statistically meaningful analysis. Other than the omission of the survey, 32 the study proceeded as before, but with the expectation that game adoption would be similarly low, and the real focus of this cohort would be collecting participants for interviews. As with all other trials, the game was introduced as a “fun” study aid containing information provided by the instructor, and as part of a research project that was investigating the design of educational games. Both versions of the game were made available to this cohort simultaneously, and they were encouraged to explore both versions with a prompt that there were some differences between the versions and that the researchers were interested to see which version of the game they used more frequently. No mention was made of a grade incentive for using the game. Interactions with the game under those conditions were observed as with all previous cohorts. After the first midterm, the option of interviewing for a 3% bonus mark was introduced to the class, resulting in 13 of 33 students agreeing to participate. 33 CHAPTER 6: Results—Part Two: Themes from the Interviews Winter 2024: MATH 190 Game Interactions Before the first midterm, 8 of 33 students had tried the game. After the first midterm, and the option of interviewing for a bonus mark was introduced to the class, 13 of 33 students agreed to be interviewed. By the end of the semester, 19 of 33 students had tried the game (Table 5), with many of the interview participants initially stating they were not interested in the game, but had become interested upon learning more about it during the interview process. Several of the interview participants used the game as a study aid for the final exam, after having already completed the interview and earning their bonus mark. Most of the participants tried both versions equally, but there was a slight tendency for a returning player to try the nontriangularity version first (there is no way to randomize the order of items in Moodle; the nontriangularity version appeared first in the course shell and was thus more likely to be selected first), then switch to the triangularity version and persist with it, even when returning to the game to play a new chapter; however, there is not enough usage data in this sample to make a statistically significant statement about this preference. Table 5 Winter 2024 Player Usage Data for MATH 190 5 # of Questions Attempted 35 Time Spent Playing (Minutes) 152 2 3 14 23 3 10 25 118 4 3 13 35 5 2 1 1 6 2 8 15 7 3 12 30 Student Number of Sessions 1 34 8 10 29 89 9 4 6 30 10 2 2 3 11 3 0 0 12 2 7 11 13 3 5 9 14 1 10 25 15 2 7 20 16 2 0 0 17 1 0 0 18 1 0 0 19 1 0 0 Interview Data Collection Nineteen students, across all cohorts, were interviewed in total. In order to isolate prior experience with games as an influence, the interviewer asked questions like “what kind of games have you played recently?”, and sorted the participants into played (those who played the game, n = 11) and did not play (those who did not play the game, n = 8), as well as gamers (n = 14, seven played and seven did not play) and non-gamers (n = 5, four played and one did not play), with gamers defined as someone who frequently engages in various forms of recreational gaming, including video gaming, card gaming, and board gaming. Although the focus of the interviews was ultimately to explore adoption and student perception of the game, the interview was framed as an exploration into triangularity, and participants were asked about their reactions to the game design. All interviewees gave consent to being quoted anonymously. Interview quotes have been edited to remove “word whiskers” and filler words. 35 Themes From the Interviews Triangularity vs Non-Triangularity The interviewees who played the game unanimously reported enjoying it. During interviews, the interviewer described the differences between the versions and interviewees unanimously, both those who played and those who did not play, indicated that they thought the triangularity version was or sounded preferable, but no meaningful statistics were gathered to support that conclusion. One interviewee, who had played both versions, described their reaction: I like the second version … because you got to choose between the hard path and the easy path. It was more control over what you wanted to be tested on and what you didn't. And then there was kind of the two paths, with a lot more of what you wanted at that moment. So, I kind of like that. Another described the inclusion of risk as a motivational factor: I think that I like that idea of adding in like the little bit of risk because I feel like that's so important because you don't know. I find that that's a good kick in the butt sometimes to actually need to try and do things. Enjoyment, Motivation, and Perception of Content Data from interviews were organized into “number of comments” and categorized by theme to visualize the opinions from the interviewees (Table 6). Table 6 Comments related to Motivation or Perception of Content Theme – Motivation and Perceptions Played (n = 11) Did not play (n = 8) Total comments 101 15% 39 10% Felt/Anticipated motivation 36 Perceived the game as a break Liked having a goal other than ‘study’ Found the game simplified studying Felt the game highlighted competency Felt/Anticipated “fiero” when answering questions “Fun math” – changed perceptions Found the game stressful/frustrating Anxiety/avoidant of math content Doubts about personal ability with math 27% 5% 16% 15% 6% 5% 1% 4% 7% 10% 8% 18% 3% 15% 36% Note: Percentage of total number of comments about experience or expectations within category regarding motivation and perceptions. Many of the interviewee comments supported that the students enjoyed the game, highlighting that the game format felt motivating for them: If it's a game like that, I wouldn't even consider that studying for my math course; it's just like training my brain, you know, keeping it sharp. Like a type of physical exercise. … I would say it's more motivating; it doesn't feel like as much as something you have to do, rather more something that you want to. You know, you're more motivated to do it. Some of those who played the game did express that they felt it motivated them to interact with the content and made the process of learning math more fun: It made my brain engage. It was fun. There was a motive to it. Like, you were trying to get all the questions right. You're trying to go fast. It's competitive. And it was on a computer. So, many things better than reading a textbook. … it was competitive, it was easy to access. It was based on math that I was trying to get good at, and so, when you enjoy getting good at something, it gets fun. One student indicated that the narrative was a motivational factor for them: When you came in and said, ‘It's time for interviews!’ I thought, ‘But I didn't finish the game! I haven't fixed the spaceship yet!’ So that night I went in and I was like ‘is it still there?’ And, whew, it's still there. 37 Several of the interviewees who did not play the game expressed that they felt it probably would be motivating, despite not personally using the game to study. Most players (27% of all comments from players) said that they found the game to be a break from traditional studying, giving them a chance to change their routine while still absorbing the content, and even some who did not play (10% of all comments from those who did not play) expressed an expectation that it would feel like taking a break. Many also expressed that they enjoyed that the narrative in the game offered a goal other than simply solving practice problems. Only one participant indicated that the game was frustrating, largely because they were an older (>50 years of age) non-gamer who was unfamiliar with digital interfaces in general. Doubts about math ability dominated the comments from those who did not play, representing 36% of all comments made from that group, followed closely by comments about math anxiety and avoidance of math (15%). The second largest category of comments from those who did not play was about how they expected the game to highlight areas of competency (18%), but these comments were often paired with comments about how they were anxious that this would make them feel bad about their own ability. Theme and Immersion in Narrative Gamers unanimously liked the theme and indicated that they felt immersed in the narrative. Many of the non-gamers who played also indicated that they enjoyed the inclusion of the narrative elements. Only one non-gamer indicated that the narrative had complicated the experience and that they would have preferred a simpler experience that focused on studying elements without any extra complications, but this interviewee also struggled with the game experience due to a low familiarity with digital interfaces. Other than this one participant, the sentiment among those who played the game was that they would have preferred more depth and 38 complexity to the game experience, adding more elements to disguise the math problems and thoroughly incorporate them into the game narrative. Many participants, unprompted, pointed to the theme as something they enjoyed. One participant said “I really like the spaceship because I really like science fiction, and I like the universe, so I was imagining myself having to do this”, which indicates that the student was finding elements of immersion while practicing math within the game. Others commented on how they enjoyed the inclusion of a narrative: I thought it was really cool. It was better than I thought it was gonna be. I was like, oh, this is cool. This is not just like a normal generic game that you would see anywhere else; it's something different. I loved the storyline. It takes away from how "serious", in air quotes, that math is or seems to be. These comments highlighted how the participants found the theme to be an unexpected element that challenged their perception of a traditional educational game: I liked it. I thought it was fun. I liked the storyline a lot. When I first opened it, I was like, 'Oh sick! This is actually like a storyline!' It's not just like, ‘here's your question, answer it’. So, I thought that was fun. It's a nice way to navigate through it. All of these comments also underline how the perception of a traditional educational game was that it would be simply a series of practice problems with few other game mechanics: I like the format of it. I like the concept; the idea that you're on a ship and each one you get builds extra time. I like the fact that wasn't just simply ‘do the questions’; that there is a background concept to it. 39 Interviewees indicated that the narrative enhanced the experience, using statements such as “Without [the ship] it's not really much of a game, it's just a digital flashcard, so it's gotta have some flair to it.” Student Expectations and Preconceptions Preconception of Game Design and Traditional Educational Games All of the participants expressed an expectation that the game would follow a traditional “flashcard” educational game format, and mentioned this frequently in the interview (32% of all comments from those who played, and 22% of all comments from those who did not play; Table 7). Many students who did not play had an expectation that the game would have poor production values, which factored into their decision to not try it: My conception around things like games and media and videos … it's probably not going to be very good unless someone really worked hard on it. I have the same feeling about games. I feel like a game could be really good as long as somebody actually worked really hard on it, but I haven't come across that many things that have actually been that helpful. So, my assumption with games is it's going to be like a “half-assed” thing. Is it really worth my time? Because you have to have to have some of those development skills, whether it's animation or the software or the programming. Table 7 Comments Related to Expectations of Students Theme - Expectations Played (n = 11) Did not play (n = 8) Total comments 31 26% 32% 10% 29% 32 22% 22% 22% 6% 16% Expected “boring” Expected traditional educational game Expected “kiddie” game Expected entertainment game “Won’t be fun” because of math content 40 Assumed low production values Total comments Expected “boring” Expected traditional educational game Expected “kiddie” game Expected entertainment game “Won’t be fun” because of math content Assumed low production values 3% 13% Gamers (n = 14) Non-gamers (n = 4) 43 23% 28% 16% 5% 21% 7% 20 25% 25% 15% 25% 10% Note: Percentage of total number of comments within category regarding expectations. This table represents two methods of sorting the same data, not two separate sets of data. Another gamer interviewee used game development terms from the entertainment industry: I was expecting an early 2000s computer game, kind of step-by-step math problems type of game. Not sure the best way to describe it. It’s just my expectations with budget and abilities to create games. Like, obviously it's not a triple-A developer game. Most of the participants also expressed skepticism that a game involving math could be fun, which did factor into the decision to not play in many cases. When asked what they expected when they first heard about the game, many of the interviewees said that they expected it to be much simpler than it was, with many interviewees specifically stating that they expected it to be boring. When asked what they expected the game to be like before trying it, one participant stated this: “Honestly, probably boring. Which I hope is not rude, but I just honestly don't know what else it would look like.” Many of the interviewees expressed surprise that it was a “real game”: I thought it was just gonna be like, a simple matching game or something really, really simple. I didn't think it would actually be that space thing that I ended up trying. I just thought it was just going to be... boring. I didn't think it was actually going to be like a real "game" game. 41 Many participants stated that they expected it to be more of a flashcard or problem-set style of game without an associated narrative: I was kind of expecting it to just be a sheet of paper with a bunch of different questions on it. I found it was a lot more of a fun version than I was expecting. Perceptions of Games as a Hobby and Gaming Purpose There was a strong sentiment from those who did not play the game that they did not feel that games were an appropriate study tool (Table 8). In the case of gamers, there was a perception that games are a means of relaxation, and introducing a study game is a pollution of their relaxation time and hobby: No concerns to its effectiveness. But for me, the idea of video games… it's an escape. Very much a way to get away from school, if I have the time to play it. And I don't know why, but the idea of using games to study feels contradictory to what I use video games for. Table 8 Comments Related to Perceptions of Games and Gaming Theme – Perceptions of Games Played (n = 11) Did not play (n = 8) Total comments 4 13% 6% 6% 12 31% 25% 19% Gamer (n = 14) Non-gamer (n = 5) 12 25% 31% 19% 4 19% 6% Do not see games as a study tool Recreation/Studying pollution Games are not for adults Total comments Do not see games as a study tool Recreation/Studying pollution Games are not for adults Note: Percentage of total number of comments (n = 16) regarding perceptions of games and gaming. This table represents two methods of sorting the same data, not two separate sets of data. 42 Some of those who played expressed this same sentiment but in a positive fashion, indicating that they enjoyed bringing their hobby into their study time because it helped to reduce their anxiety levels when dealing with math content: It's behind the screen, so if I succeed, it's not that big of a deal, whereas… I don't know. For me, sitting in a classroom, taking a pen, and then writing my name, student number, and scrambling to write everything… I feel that there is so much more pressure on it than just being behind a screen and then typing and clicking. Typing and clicking for me is a lot more of a game and I do associate my computer with relax time, whereas everything that's pen and paper is a lot more for studying time. Perceived Usefulness and Utilitarian vs Hedonic Perceptions One factor that may have affected students’ willingness to try the game was whether they believed it would be useful. Many of the interviewees indicated that they were uncertain whether investing time into the game would assist them in their coursework, or if they should spend that time on traditional study methods instead. To investigate these trends, comments from the interviews were categorized into “utilitarian” (addressing the usefulness of the game, particularly in regard to studying) or “hedonic” (addressing the enjoyment of the game). Both those who played and who did not play made more comments about the hedonic aspects of the game, but there were only slightly fewer comments from a utilitarian perspective (Table 9). Table 9 Utilitarian vs Hedonic Comments Utilitarian vs Hedonic Evaluation Played (n = 11) Did not play (n = 8) Utilitarian Comments (n = 149) 23% 34% 17% 26% Hedonic Comments (n = 226) Note: Percentage of total number of comments (n = 375) 43 The comments were then divided into gamers and non-gamers to examine the trends. Both gamers and non-gamers had more hedonic comments than utilitarian, whether they played the game or not, but both categories were well represented (Table 10). Table 10 Utilitarian vs Hedonic Comments by Player Category Utilitarian vs Hedonic Evaluation Utilitarian Comments (n = 149) Hedonic Comments (n = 226) Played (n = 7) Gamers Did Not Play (n = 7) 15% 23% 15% 23% Non-Gamers Played Did Not Play (n = 4) (n = 1) 7% 11% 2% 3% Note: Percentage of total number of comments (n = 375) The comments were further sorted into positive or negative categorization (Table 11). Table 11 Utilitarian vs Hedonic Comments by Type Utilitarian vs Hedonic Evaluation Gamers Played Did Not Play (n = 7) (n = 7) Non-Gamers Played Did Not Play (n = 4) (n = 1) Number of Utilitarian comments 57 85 50 7 73 12 88% 12% 86% 14% 28 43 21 7 28 15 75% 25% 65% 35% Number of Hedonic comments # Positive Utilitarian comments # Negative Utilitarian comments # Positive Hedonic comments # Negative Hedonic comments Positive Utilitarian comments Negative Utilitarian comments Positive Hedonic comments Negative Hedonic comments 51 89 15 36 33 56 29% 71% 37% 63% 8 14 0 8 3 11 0% 100% 21% 79% Note: Percentage of total number of comments within category. Gamers who played the game (n = 7) had many positive comments (n = 123) and few negative comments (n = 19). This group emphasized how much they enjoyed having a novel way to study (utilitarian) and how the narrative of the game enhanced their immersion (hedonic) and 44 aided in their recall of the material (utilitarian). The negative comments from this group largely focused on a dislike of the math content of the game, rather than the game experience itself. Gamers who did not play the game (n = 7) had more negative comments (n = 92) than positive ones (n = 48). Positive utilitarian comments from this group primarily centered around perceiving the game as a form of applied practice, but the negative utilitarian comments focused on a perception that games are an inappropriate method of studying at the university level and that there was no trust that the game would be useful, so the students chose traditional methods of studying instead. Positive hedonic comments from this group suggested that they were able to visualize a game that they might consider to be fun, but they needed assurance that the game would be worth their time before they would engage with it. The negative hedonic comments from this group suggested that they typically use games for recreation or relaxation, and they viewed this educational game as a pollution of their free time, or that, because of previous experiences, they had an expectation that this game would be low quality or juvenile and not worth their time. Many negative hedonic comments centered on the math content of the game and doubts that any game involving mathematics could be fun, as well as anxiety around math content and doubts about personal ability to succeed in the game due to the math content. Non-gamers who played the game (n = 4) had more positive comments (n = 49) than negative comments (n = 22). Similar to the gamers who played, this group highlighted how they found the game useful for studying and that they liked how it simplified the studying process. Unlike the gamers group, the non-gamers group did not have hedonic comments about the narrative or immersion; instead, their positive hedonic comments focused on how the added challenge of the game made the process of studying into an interesting challenge that alleviated 45 boredom. Negative utilitarian comments from this group mostly centered on how the game was not a proper way to study at university and how they did not trust that it would be useful. Non-gamers who did not play the game (n = 1) had mostly negative comments (n = 19) and few positive comments (n = 3), although it should be noted that there is only one participant described by this group. All of the utilitarian comments were negative from this participant, and primarily focused on how games were not perceived as useful for studying, as well as a sentiment that there is already enough time spent on electronic devices in school so they would prefer to spend their study time away from devices. All of the positive hedonic comments from this participant were about how their expectation was that the game would be a traditional “flashcard” style game and that the described narrative sounded more interesting to them, but not interesting enough to convince them to experiment with the game. All of the negative hedonic comments were about previous experiences with educational games in elementary school and how those experiences left them wanting to avoid educational games entirely, and how the math content of the game could not possibly be fun. Avoidance of Math Content There were a large number of negative hedonic comments that related to the math content of the game and being anxious or avoidant of the game purely because the game contained math content, or doubts about personal ability surrounding math, which were entirely external to the game or the design of it. There were also a few utilitarian comments related to perceived usefulness of studying in general that did not reflect on the game itself (e.g., a perception that the act of studying itself is a waste of time, so therefore a game designed to help you study is a similar waste of time, no matter how well-designed the game might be). Removing the 46 comments related to math and studying avoidance reveals how much of an influence the perception of math had on the negative hedonic perceptions of the game (Table 12). Table 12 Utilitarian vs Hedonic Comments by Type – Math Comments Separated Utilitarian vs Hedonic Evaluation Played (n = 7) Number of Utilitarian comments Number of Hedonic comments # Positive Utilitarian comments # Negative Utilitarian comments # Positive Studying comments # Negative Studying comments # Positive Hedonic comments # Negative Hedonic comments # Positive Math comments # Negative Math comments Positive Utilitarian comments Negative Utilitarian comments Negative Studying comments Positive Hedonic comments Negative Hedonic comments Negative Math comments 57 85 50 7 0 0 73 4 0 8 88% 12% 0% 87% 5% 9% Gamers Did Not Play (n = 7) 51 89 15 33 0 3 33 21 0 35 27% 65% 6% 37% 24% 39% Non-Gamers Played Did Not Play (n = 4) (n = 1) 28 43 21 7 0 1 28 8 0 7 75% 21% 4% 65% 19% 16% 8 14 0 7 0 1 3 4 0 7 0% 88% 13% 21% 29% 50% Note: Percentage of total number of comments within category. A negative perception of math and math content comprised a majority of the negative hedonic comments for every group except the non-gamers who played the game, who split their negative hedonic comments equally between math content and perceptions of how games are not an appropriate study method at the university level (Table 12), which could be a sample size issue since only four participants are described by this group. When the comments are sorted by positive and negative (Table 13), it is apparent that the positive comments were primarily hedonic, and the negative comments were primarily utilitarian. Non-gamers who played had more non-math negative hedonic comments than 47 gamers. Math comments dominate the negative hedonic comments for all categories of users, except for non-gamers who played, but there are likely sample size effects. Table 13 Positive vs Negative Comments by Type – Math Comments Separated Utilitarian vs Hedonic Evaluation Played (n = 7) Number of Positive comments Number of Negative comments # Positive Utilitarian comments # Positive Hedonic comments # Positive Studying comments # Positive Math comments # Negative Utilitarian comments # Negative Hedonic comments # Negative Studying comments # Negative Math comments Positive Utilitarian comments Positive Hedonic comments Negative Utilitarian comments Negative Hedonic comments Negative Studying comments Negative Math comments 123 19 50 73 0 0 7 4 0 8 41% 59% 39% 17% 0% 44% Gamers Did Not Play (n = 7) 48 92 15 33 0 0 33 21 3 35 31% 69% 41% 17% 3% 39% Note: Percentage of total number of comments within category. Non-Gamers Played Did Not Play (n = 4) (n = 1) 49 22 21 28 0 0 7 8 1 7 43% 57% 27% 36% 5% 32% 3 19 0 3 0 0 7 4 1 7 0 100% 47% 6% 6% 41% 48 CHAPTER 7: Discussion—Part Two: Assessing Adoption These interviews were an attempt to dig into the question of why students were choosing traditional study methods rather than exploring an educational game that was marketed to them as a “fun” and novel way to achieve the same purpose. What the interview comments have highlighted is that there are numerous reasons that students refused to adopt the game, but primarily, they simply had no trust in the game as a study method. Reaction Themes from the Interviews Trust Reactions Many comments highlighted an expectation that the game would be “boring”, both among those who played the game and then expressed surprise that it was not as boring as anticipated, or among those who chose not to play because of this expectation. One participant had said that they “honestly didn’t know what else [other than boring] it would look like”. These comments suggest that there is an underlying preconception that educational games in general will be boring or not worth the player’s time. While the interviews only managed to capture a subset of the students who were already willing to engage, the confirmation of this preconception means that experience with the traditional design of educational games could have influenced whether students chose to engage. Many of the interviewees stated that they had experienced educational games in elementary school. When these experiences were positive, the students were excited to try the game in their course. Unfortunately, many described these experiences as negative, either because the games they played in elementary school were poorly designed, frustrating, or boring, or because they were forced to sit at the computer and keep trying until the block of time was done. As per McGonigal (2011), if the player does not choose to play the game it becomes a 49 mandatory task, which has a different qualitative experience than voluntary play. These experiences were cited by some interviewees as the reason they did not trust the game or avoided trying it. Those who had negative experiences with educational games had very low expectations for any game in the genre, expecting it to be boring, un-fun, patronizing or childish, or generally a waste of their time and low value. Those who had previous positive experiences with games were more willing to engage with the game and expressed curiosity, but very few participants engaged with the game with high optimism about what they would experience. The overwhelming response from participants was surprise that the game was different than their preconceived expectations. These preconceived expectations were deeper in the case of gamers, who have extensive experience with games of all genres. These participants have experience with games and some understanding of game design, and recognized that an educational game is unlikely to be “triple A”, especially when presented by a graduate student who does not have access to a design studio. Even though some gamers were curious about the game, most stated that they did not feel it would be high quality and therefore would be unlikely to be worth their time, which meant they did not bother to engage with the game unless they had a significant amount of free study time to spare (which is a situation that most post-secondary students do not often enjoy). Those who found time to play expressed that they enjoyed the game and found it motivating, but overall adoption was low presumably because of these preconceptions of the game being low value. Gamers had an additional barrier in that the educational game represented a pollution of their hobby and relaxation time. Many expressed that they did not want to play the educational game because they felt it was an inappropriate use of the gaming genre, so they deliberately 50 avoided it. For these participants, even if the game had been high quality, they disagreed with the concept of educational games and somewhat resented the attempt to pander to their interests. The final “trust” reaction was that students did not trust that the game would be useful, or entertaining. If the player expected a game, they did not trust that the game would be fun because it included studying elements, or if the player expected a study tool, they did not trust that it would be as useful as a traditional study method because of the focus on or distraction of game elements. Comments from this category led to a deeper investigation into the hedonic and utilitarian elements of educational games, which are discussed in more depth in Chapter 8. Complexity Reactions There was a very clear theme of the interviewees expecting the game to be simple flash cards. This expectation of the game simply being practice problems underscores the current status of the educational game field, where many traditional educational games are primarily a “lecture in a box” or “interactive flashcards” and are lacking the game mechanics to create an engaging challenge. In general, interview comments suggested that the students expected the game to be very simple, possibly because of prior experience with educational games designed for younger audiences that, by necessity, are simple in design. Many expressed surprise that the game was more complicated than expected, or expressed a desire for it to be even more complicated. The game built for this study was actually fairly simple in design because it needed to be compared directly against a “traditional” design. The intent of the design was to isolate triangularity, so there needed to be a “traditional” educational game, and then a nearly identical game that had triangularity injected into the design. There was potential to add more game mechanics to the game, such as a deeper narrative, characters to speak to, quests to solve, items 51 to collect, or skills to level, but because triangularity was the focus, the only added game mechanic was that of choosing a more- or less-complicated path and making decisions on the level of risk to attempt for each question. Even though this design is quite simple in itself, the students still reacted with surprise that it was more complicated than they expected, which suggests that their perception of a traditional educational game is that it will be exceedingly simplistic. This expectation may have factored into a lack of desire to try the game. These expectations could have been defused by an in-class demonstration of gameplay, but it is worth noting that the default expectations of the target audience appear to be a barrier that must be anticipated in order to facilitate adoption. Social Identity Reactions An additional, and somewhat unexpected, category of reaction was that students may have avoided the game because they dislike the social category that games potentially represent. Those who self-identified as “non-gamers” often voiced sentiments like “I don’t play video games” and had strong opinions about the suitability of games as utility, expressing opinions such as “games are not serious or effective study tools”, “games are for children”, or “gamers are lazy”, with one interviewee specifically categorizing games as “for uneducated dumb people who sit in their basement all day”. These comments indicate that there could be a confound where the perception of “gamers” or the perceived target audiences of games makes some potential users hesitant to try a game because they do not want to be identified with a particular social group (e.g., “gamers”, or being seen as “playing a kiddie game”). Math Content Reactions The game designed for this study was part of a mathematics course and was openly revealed to have math content, which resulted in negative responses from the students. The 52 magnitude of the math avoidance response was unanticipated. Several of the interviewees specifically said they would have tried the game if it was not math content, indicating that the adverse reaction to math overwhelmed any other affective response to the game. The courses featured were considered ideal candidates because the students in these sorts of courses are stepping outside of their comfort zone, and MATH 190 in particular spends time focusing on math mechanics in games and making it more fun to learn, so it was thought that a game would be a novel distraction and interesting to explore. In reality, many of these students likely view these courses as high risk, and are taking them as a last resort, looking for the “easiest” math credit they can collect so that they can move on with their degree. To some of these participants, the course represents a heightened risk of a poor grade, which could limit their access to a career path; therefore, the game may have appeared too risky, because they would be spending their limited study time on an unknown method and potentially putting themselves at even greater risk of a poor grade. This level of risk could be an example of a “too much at stake” threshold that resulted in disengagement, as described by Arrasvuori (2011). For those students, buckling down and doing traditional study methods represented the safe route through the course, and they did not entertain spending time taking a risk on an unknown method like an educational game. This result is notable because the purpose of an educational game is to motivate the student to interact with material that they may have otherwise not interacted with. Affect is known to impact technology adoption (Davis, 1989) and hedonic systems are usually intentionally designed to induce positive affect to influence behaviour (Lowry, 2013). The avoidance of the game simply because it featured math content suggests that the presence of an educational game was ineffective for that purpose in this context, which also indicates that the 53 potential lack of adoption must be considered along with resource and development costs for an educational game design project. Limitations of Part Two In addition to the limitations noted in part one, the sample size effects noted in part one were exacerbated in part two. Even though the game was presented to 117 students over five courses, only 19 students agreed to interview about their choice to play or not play the game. It should also be noted that there was self-selection bias in this sampling, and that students who had a strong aversion or who were extremely uninterested in the game would be less likely to accept the invitation to interview, and that, even though most interview responses felt genuine, there was a chance that an interviewee may have been participating merely to obtain the grade boost or gift card. Given that the sample is limited and self-selected, and certain subgroups are small, such as non-gamers who chose not to play, the findings are illustrative, not representative. The late-stage refocus to interviews also means that some interviews were conducted with participants who were several months removed from the course and/or game experience, and their memory of the game or reasoning for adopting or not adopting the game as a study method may have been less concrete than those who had been introduced to the game in the same semester as the interview. 54 CHAPTER 8: Utilitarian Evaluation of Educational Games Part one of this investigation revealed that students were actively choosing not to adopt the game as a study method for MATH 190 and XMAT at UNBC, and part two investigated those choices through interviewing the students. One interview theme that stood out was how the students were rejecting the game based on their preconceptions and perceptions of whether the game would be not fun enough (hedonic), or not useful enough (utilitarian). The majority of existing research focuses on the hedonic aspects of games, which makes sense—games are hedonic vehicles. The initial investigation of this study was also focusing on hedonic aspects: whether adding triangularity and increasing player agency by adding risk vs reward would improve adoption and motivation. The unexpected reaction from the students was that they evaluated the game for its utilitarian aspects in addition to, or in some cases despite, its hedonic value. As part of introducing the game to each course, I announced “Who wants to play a game?!”, which was typically met with interest. I followed that statement with “It’s an educational game!” which was unanimously met with disappointment. The question was intended to build interest in the research question, but it may also be a hint to the results: it is possible that once the game was introduced as “an educational game”, the students expected the game to be useful (utilitarian) rather than fun (hedonic), and began to assess it as a tool instead of a game. This sort of utilitarian assessment appears to be relatively unknown and uninvestigated in educational game research. Summary of Data Analyses: Hedonic and Utilitarian Values Data from interviews were organized into “number of comments” which were then sorted into both positive/negative, and utilitarian/hedonic categories. 55 When sorted into positive/negative (Table 13), there was a trend of the positive comments being primarily hedonic among both those who played and those who did not play, suggesting that both groups either thought the game was fun, or assumed that it was intended to be fun even if they did not play it. Those who played had more positive utilitarian comments than those who did not play, suggesting that those who played found it useful, but also suggesting that those who did not play did not expect it to be useful. When math comments were stripped from the negative comments (Table 12; Table 13), the negative comments were primarily utilitarian, suggesting that both groups had doubts about the usefulness of the game, but had a perception that the game was intended to be a hedonic experience. The game was marketed as “a fun way to study”, so the students were primed to assess it hedonically, which might account for this perception. Non-gamers had more negative hedonic comments than gamers (Table 12; Table 13). This trend could be the result of a sample size issue (only four non-gamers played and volunteered for an interview), but it could also be an indication that the non-gamers were looking for utility and felt that the hedonic narrative aspects were an unnecessary complication, while the gamers were expecting utility and discovered a more traditional hedonic game narrative— perhaps demonstrated by how several gamer interviewees mentioned their surprise at finding a “real game” or “game game”. The non-gamers had fewer negative utility comments than the gamers, possibly because they were primarily looking for utility, while the gamers were possibly looking for hedonic experiences. When sorted by utility/hedonic, the utilitarian comments were mostly positive among those who played the game (Table 12), which suggests that if they played it, they found it useful. Utility comments were mostly negative among those who did not play the game (Table 12), 56 suggesting that those who did not play the game assumed it would not be useful or worth their time. Both those who played and those who did not play had mostly positive hedonic comments (Table 12; Table 13), suggesting that those who played found it fun, and those who did not play assumed that the game was intended to be fun, even though they did not play it. The game was marketed as a “fun way to study” which may have coloured their perceptions and expectations. The majority of negative hedonic comments were about math (Table 12; Table 13). Once those comments were isolated, there were very few negative hedonic comments. The negative perception of math appears to represent a separate hurdle that must be considered when designing an educational game with math content: no matter how well-designed a game is, the initial perception is likely to be that a game containing math content will be inherently un-fun. The largest amount of non-math negative hedonic comments came from non-gamers who played the game (Table 13), which could be because they hoped for more utilitarian aspects, or because they were inexperienced with or disinclined to use video games, making the game an unfamiliar and perhaps uncomfortable experience for them. It is possible that this group hoped for a tool, not a game. Overall Expectations and Assumptions It appears that when I promoted the program as “a game”, the students made hedonic assumptions, but when I promoted it as “an educational game”, their expectation was “interactive flashcards”, which is more of a utilitarian viewpoint. Many of the comments received in interviews suggested that “interactive flashcards” was the default expectation from both gamers and non-gamers. Those who played the game may have played either because they expected fun (hedonic), or because they expected flashcards (utility), or, optimistically, because they expected fun 57 flashcards. Those who did not play may have decided that the game was not worth their time, either because they dismissed the hedonic side (e.g. “it’s going to be flashcards, and it’s going to have math, and neither of those things are fun”), or because they valued other utility (such as traditional, known methods of studying, or not spending time on studying at all) higher than any potential utility from the game. Expectancy-Value theory (Wigfield & Eccles, 2000) describes four core values that might be helpful in explaining these reactions. Attainment value is the importance of doing well on a task. If the students believe they will not do well in the game, either because of math efficacy or because of low familiarity with gaming or low gaming skills, their attainment will be low. The intrinsic value is the enjoyment of doing the task, which will be low if the students believe the game is not fun because of prior experiences with educational games, expectations around math or math efficacy, or expectations around games and gaming. The utility value is how useful they view the task and how it fits into their future plans, and the utility value of the game hinges on whether they believe the game will boost their grade or not. Finally, cost is how much time they have available. If the student has little time or poor math efficacy, the cost of the game will be perceived as too high, leading to low adoption or adoption of lower cost methods. The interviews highlighted that students are familiar with traditional “flashcard” educational game designs from elementary school, and they do not expect anything different when presented with something branded as “an educational game”. The comments from these interviews suggest that educational game designers should consider the initial reactions of students and whether they are assessing the game as hedonic or utilitarian. It appears that if the student does not believe the game will be useful, their hedonic assessment becomes irrelevant, and adoption will be low. There is currently very little research into this utilitarian assessment. 58 Existing Literature: Exergaming as an Analogue While research is currently sparse on educational games—particularly educational games with adult users—exergaming (games designed to promote physical activity (Kari, 2017)) serves a similar purpose: to motivate a target audience to do an action (studying or exercising) through gaming. Exergames are more likely to target an adult audience, and there is some research into utilitarian adoption of the genre, making exergaming a potential analogue for educational games. In a dissertation, Kari (2017) outlined a list of reasons to adopt exergames, including facilitators such as “competition”, “greater variety of music in dance games”, “enjoyment”, and “not constrained by typical barriers to exercise”. Kari also outlined barriers such as “boredom”, “technical problems”, “changes in living arrangements”, and “auditory nuisance” (Kari, 2017, p. 48). Kari had not sorted these reasons into utilitarian/hedonic categories as part of their investigation, but upon categorizing them for comparison purposes, it appears that Kari discovered largely positive hedonic reasons to adopt and largely negative utilitarian reasons to avoid, which parallels the students’ reactions to the educational game as observed through the interviews. Kari also observed that “lack of interest” (p. 54) was the primary reason for not playing an exergame, but there was no information on whether the users felt this way because they considered the game “not fun” or “not useful”. The second most common reason for not playing an exergame was “prefers other methods” (Kari, 2017, p. 54), which was also observed with the educational game, but it is unclear if users preferred other methods because they believed the game would be less useful, or because they believed the game would not be fun, or if was is a combination of both hedonic and utilitarian reasons. Comments from the interviews suggest that both the game and traditional studying methods were commonly viewed as “boring”, but many students chose traditional studying over playing the game, which suggests that even 59 when both activities were considered “boring”, they still did not value the game highly enough to give it their time. Kari (2017) also stated, “the responses suggest that the playing of exergames is not perceived as having significant effects on physical fitness” (p. 151), with a majority of the participants indicating that playing the game had no effect on their physical fitness. This comment suggests that the perception is that the game is not useful; it is mainly viewed as a trinket or toy rather than a serious method of exercising. There could be a parallel to these comments with the interview comments that suggested that the game would not be useful or appropriate for studying, and it may help to explain why traditional studying remained a preferred method for many students; if the potential users do not believe the game will be a hedonic experience worth their time, and also do not believe the game will be a serious utility with significant effect, they are likely to choose familiar methods that are known to be useful. Kari (2017) had noted that exergames were often not strenuous enough for serious athletes, but raising the difficulty to that level would make the game too frustrating for the groups that the game is intended to motivate, which could contribute to this perception of the game not being a serious method of training. Educational games face a similar conundrum, where the difficulty curve needs to closely match the skill level of the user to avoid boredom (the content is too easy) or frustration (the content is too difficult) (Malone, 1982). In my review of post-secondary educational games (Appendix A), I did not encounter any well-designed educational games that effectively modulate their difficulty to the user’s skill level. If the students are familiar with games that do not scaffold effectively, there is likely little trust from the students that their experience with a novel educational game will not be boring or frustrating. Critical Incident Theory 60 Kari (2017) also had observations about “critical incident theory” where he observed that adoption of the game was largely driven by hedonic reasons, like enjoyment, but continuance of usage depended on the user having a positive experience with the game—a positive “critical incident”— with the hedonic aspects having greater weight on their positive experience than the usefulness of the game. The utilitarian aspects of the game were important for the intended outcome of the game, but Kari observed that users were more likely to continue if the game had a mix of utilitarian and hedonic aspects. In other words, the users were most likely to adopt the program if they thought it would be fun, but were more likely to keep using the game if it was fun and useful. If they felt the game was fun but not useful, they were likely to consider it frivolous or insignificant (albeit amusing) and not persist with use, and if they felt it was useful but not fun, they would be likely to return to known useful methods that they were more familiar with. Kari’s conclusion was that exergame design needed to balance utilitarian and hedonic aspects to be effective. This conclusion is likely applicable to educational games as well. Critical incident theory (Flanagan, 1954) seems relevant in reference to experiences with educational gaming in elementary school. The frustration of these sessions could have resulted in a “negative critical incident” that coloured student perceptions of the content and the educational game format. Even in a more moderate case where the student had no strong feelings about their time in an educational game computer lab session, being exposed to a traditionally-designed “interactive flashcard” game is a critical incident that could contribute to an underlying expectation that educational games are utility-first and “not fun”, changing the perceptions, expectations, and initial assessments of the user when presented with other educational games. 61 CHAPTER 9: Research Gaps and Future Research This study set out to investigate whether triangularity should be prioritized in educational game design, particularly when considering design for the post-secondary market, which has been under-utilized. What has been revealed through this investigation are knowledge gaps and potential research directions that could have implications for educational game design in general, not just limited to post-secondary markets. This chapter outlines potential areas of research and suggestions for future studies that could potentially contribute to our understanding. Potential Themes to Investigate for Future Research Utilitarian Evaluation vs Hedonic Evaluation in Program Adoption There is currently very little research into utilitarian evaluation in educational games, and since the primary targets for most educational games are elementary-aged children (Richtel, 2005), there is even less information about these expectations and evaluations in adult users (i.e., the target audience for post-secondary education). Kari (2017) had observed this lack of research for the exergame field, stating that motives for using exergames are still relatively unknown (p. 16), and had suggested that "future studies could benefit from using some additional criteria to group the data (e.g., criteria concerning users’ hedonic/utilitarian motives)" (p. 71). Kari also stated that very few studies have looked at the hedonic/utilitarian split and that future studies should consider “simultaneous dual-purposed” hedonic/utilitarian use of information systems (p. 65). Future research into educational game design should also take these factors into account, and the Information Systems field might want to consider grouping programs like exergaming and educational gaming together if the parallels between the two prove to be consistent. At the time of writing, most of the research into educational games appears to be focused on the hedonic experience, and there appears to be very little research into utilitarian evaluation 62 of games, which presents a category of opportunity for future researchers to explore. Even though Kari (2017) investigated utilitarian and hedonic aspects of exergames, their research did not focus on utilitarian vs hedonic evaluation when examining reasons not to play a provided game. The split between “this game will be fun” and “this game will be useful” seems to be an important aspect to consider when designing in either an educational or exergame genre, especially since both this study and Kari’s results suggest that both aspects are important to adoption and persistence of use. Snap Evaluation and Potential Branding Issues It is possible that there will be a difference in expectations between a program presented as “a game” (where students tend to expect a hedonic experience) vs “an educational game” (where students will apparently expect a utilitarian experience). A mere phrasing change in how the game is introduced to the students might make a difference in the perceptions they bring to their interactions and could reveal insights into how to market this genre. These perceptions could be investigated with a comparison study. If it is revealed that simply naming the program “an educational game” instead of “game” changes reactions to it to such a degree, it will underscore the branding issue that the educational games genre is facing. Perception of the Purpose of Games Educational games have an inherent assumption of “gamers like games, therefore if we put education into a game, maybe gamers will like education!” (McGonigal, 2011). On the contrary, some of the gamers indicated that the educational game represented a pollution of their hobby and relaxation time, and that they resented attempts to pander to their interests, creating an aversion response. A more delicate approach that incorporates desirable elements of gaming without targeting the gaming hobby might be more successful, but a deeper investigation into 63 this category of reaction would be required to determine the best approach, especially given that some gamers responded positively to the attempt to bring their hobby into their study time. Many interviewees expressed opinions about the suitability of games as utility programs, and there appeared to be a trend of social identity modulating their responses. Introducing the game as a “game” might alienate potential users who view games as intended for children, or as something associated with or specifically for gamers. Similar to the “game” vs “educational game” reaction, it could be worth investigating to see if a program marketed as a “tool” received a different response than one marketed as a “game”, as it could affect the market viability of this genre and represents a potential avenue for research to develop strategies to mitigate these perceptions. However, there is a two-prong consideration, as marketing it as a “tool” will almost certainly affect utilitarian evaluation, which could have additional evaluation effects. As with the other potential branding issues identified, the educational games genre has some work to do in order to improve its image and become cemented as a commonplace and acceptable alternative to traditional methods, particularly with adults. Reactions to Math Content The negative affective reaction to mathematics content appears to be particularly potent, and represented an unanticipated barrier that overwhelmed any positive affective reaction the game may have elicited; therefore, it might be worthwhile to try a similar study with non-math content to see how responses differ and to get a baseline measurement of user response without the math confound. It is possible that educational games can enjoy wide adoption and be effective and engaging study tools for a non-math course, although a positive result in such a study would mean educational games are ineffective at their intended purpose if the content is mathematics, which would mean math educators still, probably unsurprisingly, have additional 64 challenges to grapple with. Mathematics education appears to be a different category of challenge—one that needs to be untangled from the general challenges that game adoption faces. Interviewees also expressed a desire to have the math content “hidden” behind more game mechanics, so it could be an interesting comparison to develop a more robustly-designed game that incorporates the math content into the goals of the narrative, but is marketed as a “fun game” with no mention of the content. One interviewee said this: It really got me thinking like, oh my gosh, I literally do math every single night and I hate math. It's my least favourite subject, but I’m doing it willingly, for fun, every single night before bed, like the last thing I do. So that's like ‘huh, Maybe I don't hate math.’ The sentiment of “maybe I don’t hate math” is a “holy grail” for this genre. Having the math hidden so that the player must learn the content to achieve game goals, having fun and being engaged playing a game while also applying the knowledge that is intended for them to learn, then recognizing that the content was useful, is the best-case outcome for an educational game. Unfortunately, designing such a game would be an undertaking in terms of both design and programming, which also represents a core challenge for the educational gaming genre. Educational Games vs Simulators Another interesting approach to educational gaming, and one that could also help to distinguish educational games from exercise games, could be to delineate the difference between “an educational game” and “a simulator” where the user is applying their knowledge in a safe and structured test environment. The comments from the interviewees indicated that the students felt the best use for an educational game would be applied practice, where they were forced to apply their skills to achieve some purpose, not just to solve practice problems for the sake of practice. Ryan et al. (2006) hypothesized that simulations can provide an avenue to practice 65 safely, fostering a sense of autonomy and control, leading to a feeling of accomplishment and perceived competence, which, in turn, leads to heightened satisfaction. Simulators provide a perfect environment for such work, but there is a lack of research into the hedonic side of simulations. If it is true that the best environment for persistence in a utility game is to have content that is useful and fun (Kari, 2017), then it might be worthwhile to investigate where the line is that flips a simulator from “utilitarian” to “hedonic” in a user’s mind. This sort of investigation could be done by designing simulations with varying forms of narrative, to find the thresholds that boost the user’s hedonic evaluation of the program. Much like other proposed studies, this investigation would require a significant amount of effort in terms of designing and programming the test game/simulation. Resiliency in Gamers vs Non-gamers Comments from the interviewees suggested a theme of gamers viewing failure as a challenge, where they feel competitive and want to do better, expressing sentiments like “I had to keep trying until I fixed the ship!”, while non-gamers expressed sentiments like “I didn’t want to try the game because I suck at games”, “I’m bad at math”, or “I don’t have a ‘math brain’” and refused to initiate, or would quit upon their first failure. Through these comments, it seemed as though gamers were displaying more resiliency than non-gamers. Plass et al. (2015) described “ability to fail gracefully” as an important part of learning. It would be interesting to investigate these trends and evaluate whether gamers were resilient to begin with, or if their experiences with gaming (where failing and restarting a level is a large part of most gaming loops) helped to build resiliency. It could be that gamers have learned through prior experience that a game or simulation is a safe place to experiment and fail while non-gamers remained anxious even in these “safe” situations because they do not recognize them as safe. If relevant, educational games 66 as a form of resiliency training could be a valuable direction for the genre to pursue, but adoption of the game will remain a challenge, especially in the face of content-driven anxiety. Narrative Enhancing Immersion Many of the “gamer” interviewees cited how they play games to immerse themselves and “escape” or “forget” their current situation, which aligns with research in the field (Ryan et al, 2006). Research suggests that immersion aids in absorption of knowledge (Csikszentmihalyi, 2014; McGonigal, 2011), so an investigation into the outcomes of a serious game (either in retained knowledge or in persistence with health habits) might want to consider immersion and create a comparison between a “focused” game with traditional task-focused design compared to one with an “immersive” design that hides the tasks behind a narrative. If it is revealed that immersion enhances the outcomes of a game, it could have a profound effect on the design of future games in the genre. Another potential point of consideration in this area is skill-building feedback. Traditional “serious game” design tends to regularly break immersion to remind the user that they are studying or exercising, either by awarding badges describing those tasks or by building narrative that is entirely focused on that aspect of the game. It may be valuable to investigate whether the potential knowledge-absorption benefits of immersion outweigh the benefits of breaking immersion to offer advice, or to look for a threshold where feedback can be incorporated with minimal disruption to immersion. If immersion is found to enhance knowledge absorption, and techniques can be developed to provide skill-building feedback without breaking immersion, the benefits of both could potentially enhance a game. 67 Cognitive Load and Exam Preparation Some participants indicated that they liked that the game was doing the decision-making of practice for them; they did not have to choose which questions to practice on, because the game would provide a random shuffle of questions to work through, providing a mix of concepts they had mastered and challenging concepts that they needed to focus on before their exam. It is therefore possible that the game was reducing their cognitive load by taking the work out of choosing what to study, leaving more capacity to focus on the math itself. As suggested by Deslaurier et al. (2019), increased cognitive effort required for learning activities can potentially result in the students feeling that they have less mastery over the material, which could mean that an educational game that provides any reduction in cognitive load could feel beneficial to the student, even if the ultimate effect on the learning outcome is marginal. An investigation into the cognitive load of studying, and whether the presence of a studying game or studying utility (even if it has no game elements) influences that load, could be helpful for future program designers. There were also a number of comments suggesting that the time constraints within the game were helpful for preparing for exams, which was one of the intentions behind the time limit on the “engine question” in the game design. Having real-world experience with how long it can take to complete similar questions, as well as experience with how they will react to the imposed time limit, could potentially help a student prepare for the exam environment. It would be interesting to have an investigation into how effective a simulated exam environment is for exam preparation, and a comparison of the effectiveness of a true simulation (a pure practice exam) compared to a gamified simulation (flashcards with gamified elements like badges and progress meters) or true game where the exam practice is hidden behind a narrative. Results from this sort of investigation could inform future educational game design. 68 CHAPTER 10: Conclusion This study began as an investigation into whether triangularity, a mechanic that is not well-researched in current educational game design, should have a more prominent focus. The answer to that research question was a tentative “maybe”, because those who played the game unanimously indicated they preferred the triangularity version; however, there were so few students adopting the game that it was apparent that the question of “why are students avoiding the educational game?” needed to take priority. This investigation has revealed a gap in educational game research. Most of the current educational game design research focuses on hedonic aspects, but the interviewees in this study indicated that they were evaluating the game for both hedonic and utilitarian usage. The students were looking for both a fun experience and a useful tool. If the game was perceived as fun, but not useful, the student might adopt if they had free time, but usage would not persist. If the game was perceived as useful, but not fun, adoption would depend on whether the students already had an established utility method. If the game was perceived as not fun or useful, the students would not adopt the game at all. The balance of hedonic aspects and utility of educational games, particularly at the post-secondary level with adult audiences, needs to be investigated further. The perceptions and expectations of the students were coloured by previous experiences with the educational game genre. The comments from the interviews suggested that educational games have a branding issue; when a program is introduced as “an educational game”, students assume there is low hedonic value and start to evaluate it as a tool. Prior experience with poorlydesigned educational games and critical incident experiences with enforced “play” in elementary school may have precipitated these perceptions. Educational games are not a “magic bullet” to make learning fun, and forcing a student to play one changes it from a game into an assignment 69 (McGonigal, 2011), potentially harming perceptions of the entire genre. It seems important to consider not just the design of educational games, but also the formative first experiences of potential users and how the games are being deployed to them, particularly in elementary school. At present, the majority of research into educational games is focused on children, which means there is a dearth of information about a post-secondary or adult audience. Exergames and educational games appear to have many parallels, so it might be worth grouping them together to look for other analogues. If these two genres overlap, exergame research might help us bridge some of the knowledge gaps we have for educational games, particularly in regard to adult audiences or utilitarian evaluation. The comments from interviewees suggested that students have come to expect stark utilitarian design in educational games, to the point that they fail to engage. With the current lack of adoption due to perceptions of the genre, it could be argued that educational games are not worth the development time and resources they require, regardless of potential benefits. It seems important to better understand the line between “tool” (utility, not necessarily fun) and “trinket” (fun, but not necessarily useful) to find the recipe for a “fun tool”. Understanding the balance of utilitarian and hedonic mechanics and how to incorporate them into game design is likely the first step to improving the perception and adoption of educational games and making this genre viable at the post-secondary level. 70 References Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., De Freitas, S., Louchart, S., Suttie, N., Berta, R., & De Gloria, A. (2015). Mapping learning and game mechanics for serious games analysis. British Journal of Educational Technology, 46(2), 391-411. https://doi.org/10.1111/bjet.12113 Arrasvuori, J., Boberg, M., Holopainen, J., Korhonen, H., Lucero, A. & Markus, M. (2011, June). Applying the PLEX framework in designing for playfulness. In Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces (pp. 1-8). https://doi.org/10.1145/2347504.2347531 Articulate Global. (2023). Articulate Storyline 360. https://articulate.com/360 Boberg, M., Karapanos, E., Holopainen, J., & Lucero, A. (2015, October). PLEXQ: Towards a playful experiences questionnaire. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (pp. 381-391). http://dx.doi.org/10.1145/2793107.2793124 Classic Reload. (n.d.). The Oregon Trail. https://classicreload.com/oregon-trail.html Csikszentmihalyi, M. (2014). Flow and the foundations of positive psychology: The collected works of Mihalyi Csikszentmihalyi. Springer. https://doi.org/10.1007/978-94-017-9088-8 Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008 Deci, E. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social Psychology, 18(1), 105-115. http://dx.doi.org/10.1037/h0030644 Deslauriers, L., McCarty, L., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the 71 classroom. In Proceedings of the National Academy of Sciences, 116(39), 19251-19257. https://doi.org/10.1073/pnas.1821936116 Djaouti, D., Alvarez, J., Jessel, J.–P., & Rampnoux, O. (2011). Origins of serious games. In Ma M., Oikonomou, A., Jain, L. (Eds.), Serious Games and Edutainment Applications. https://doi.org/10.1007/978-1-4471-2161-9_3 Drexel University. (n.d.). Digital games and game-based learning. Drexel University College of Medicine. https://drexel.edu/medicine/about/departments/institute-for-molecularmedicine-infectious-disease/administrative-centers-for-excellence/center-for-businessand-program-development/digital-games-and-game-based-learning/ Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358. Gjarnjobst, N. (2017). Oregon Trail (computer game). Minnesota Historical Society. https://www.mnopedia.org/thing/oregon-trail-computer-game Hamari, J., Shernoff, D., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170-179. https://doi.org/10.1016/j.chb.2015.07.045 Huynh, D., Zuo, L. & Iida, H. (2016). Analyzing gamification of “Duolingo” with focus on its course structure. In Bottino, R., Jeuring, J., & Veltkamp, R. (Eds.), Games and Learning Alliance, 268-277. https://doi.org/10.1007/978-3-319-50182-6_24 Jenkins, H. (2006, June 23). Fun vs. engagement: The case of The Great Zoombinis. Confessions of an ACA-FAN. http://henryjenkins.org/blog/2006/06/fun_vs_engagement_the_case_of.html 72 Kari, T. (2017). Exergaming usage: Hedonic and utilitarian aspects [Doctoral dissertation, University of Jyväskylä]. http://urn.fi/URN:ISBN:978-951-39-6956-1 Krathwohl, D. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2 Leithead, C., Lin, A., Go, N., Visutskie, K., Foxe, J. P., & Miller, A. (2023). Promoting academic integrity in virtual classrooms: A gamified approach. Canadian Perspectives on Academic Integrity, 6(1), 1. https://doi.org/10.11575/cpai.v6i1.76919 Lim, T., Carvalho, M., Bellotti, F., Arnab, S., de Freitas, S., Louchart, S., Suttie, N., Berta, R., & Gloria, A. (2015). The LM-GM framework for serious games analysis. Lowry, P., Jenkins, J., Gaskin, J., & Hammer, B., Twyman, N., & Hassel, M. (2013). Proposing the hedonic affect model (HAM) to explain how stimuli and performance expectations predict affect in individual and group hedonic systems use. Sprouts: Working Papers on Information Systems, 8(24), 1-55. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2287401 Malone, T. (1982). Heuristics for designing enjoyable user interfaces: Lessons from computer games. In Proceedings of the 1982 Conference on Human Factors in Computing Systems (pp. 63-68). http://portal.acm.org/citation.cfm?doid=800049.801756 McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. Penguin. New Mexico State University. (n.d.). Potluck Panic. https://potluckpanic.nmsu.edu/ OpenStax. (2023). Elementary Algebra 2e. Retrieved from https://openstax.org/details/books/elementary-algebra-2e 73 Peters, J. (2007, June). World of borecraft. Retrieved from https://slate.com/technology/2007/06/the-trouble-with-serious-video-games.html Plass, J., Homer, B., & Kinzer, C. (2015). Foundations of game-based learning. Educational Psychologist, 50(4), 258-283. https://doi.org/10.1080/00461520.2015.1122533 Richtel, M. (2005). “Once a booming market, educational software for the PC takes a nosedive”. New York Times. https://www.nytimes.com/2005/08/22/technology/once-a-boomingmarket-educational-software-for-the-pc-takes-a.html Ryan, R., Rigby, S., & Przybylski, A. (2006). The motivational pull of video games: A selfdetermination theory approach. Motivation and Emotion, 30(4), 347-363. https://doi.org/10.1007/s11031-006-9051-8 Schell, J. (2008). The art of game design: A book of lenses. Elsevier. https://doi.org/10.1201/9780080919171 Schoenau-Fog, H. (2011). The player engagement process—An exploration of continuation desire in digital games. In Proceedings of the DigRA Conference: Think Design Play (pp. 1-18). http://www.digra.org/digital-library/publications/the-player-engagement-processan-exploration-of-continuation-desire-in-digital-games/ Skinner, B. F. (2014). Science and human behavior. B. F. Skinner Foundation. (Original work published 1953) Vygotsky, L. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Harvard University Press. Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015 74 Appendix A: Educational Game Design In order to determine what mechanics are typically included in educational games, I searched for some more recent examples of educational games that were designed for the postsecondary level. In order to evaluate these games, I chose the Learning Mechanics-Game Mechanics (LM-GM) model (Arnab et al., 2015; Lim et al., 2015) to compare the design of the games within a consistent framework. The LM-GM Model Arnab et al. (2015) proposed the LM-GM model to merge learning mechanics (LM) (e.g.: Bloom’s Taxonomy (Krathwohl, 2002)) with game mechanics (GM). Within this framework, Arnab et al. (2015) refer to “Serious Game Mechanics” that transition a learning goal into a gameplay mechanic. The LM-GM model (Figure 2; Figure 3) can be used to assess whether a serious game is encouraging the player to move into Higher-Order Thinking Skills (HOTS), and can help suggest game mechanics to implement for the target level. Figure 2 The LM-GM model combined with Bloom’s Taxonomy (Lim et al., 2015) 75 Note: Copyright Lim et al. (2015). Used with permission. Figure 3 A game map demonstrating the LM-GM model (Lim et al., 2015) Note: Copyright Lim et al. (2015). Used with permission. 76 I used the LM-GM model as a framework to evaluate the game mechanics that were utilized in post-secondary educational games in order to determine what level of “Thinking Skills” the games might reach on the LM-GM hierarchy. The ultimate goal of Bloom’s Taxonomy is to reach the “Creating” level, where one is utilizing and applying the knowledge in novel situations, particularly if the student can use it to create their own experiences or products, or create a lesson for another. The LM-GM model equates the creating level to game mechanics such as a protégé effect, where players learn motivation for their own behaviour and can teach another these actions, and strategy/planning where the player uses their knowledge to prepare for more advanced levels. However, it was a bit unclear how some parts of the LM-GM related to creating, such as tiles/grids and status, which could both be information systems that do not require player interaction, or infinite game play, which could occur even if no learning is occurring. Triangularity fits at the “evaluating” level because it requires the player to use their knowledge to make decisions and evaluate the risk vs the reward being offered (as per the LMGM, “assessment”, “hypothesis”, and “reflection”). It could be argued that a player who is truly exercising their agency to evaluate a situation and make a strategic gameplay decision is demonstrating taking ownership, responsibility, and planning, which would indicate working at the creating level. However, it is not guaranteed that the information they are basing their strategy on is the same information that they were intended to learn, so some care must be taken when applying these labels. While evaluating these post-secondary educational games, I attempted to match the game mechanics I found within the game to the hierarchy provided by the LM-GM, and also look for evidence that triangularity was incorporated into the design. 77 Examples of Educational Game Design in Use in Post-Secondary Education Today Oregon Trail (1985) While Oregon Trail was not a post-secondary game, it is a classic example of a successful educational game and I wanted to start my investigation by evaluating it in the LMGM framework as a comparator for modern game design. Anyone who attended school in the late 1980s or ‘90s probably remembers Oregon Trail, and it can still be played today on sites like Classic Reload (Classic Reload, n.d.). Oregon Trail asked the player to lead their family along the real-world path of the pioneers of the 1800s, from Kansas to Oregon. The player had to make decisions like which supplies to bring, which routes to take, and how to deal with events along the way. Most players would not have thought of Oregon Trail as an educational game, but it featured a real setting and exposed the player to real facts from that era, as well as the various techniques pioneers used to survive, such as caulking their wagon to ford a river. Players also learned how perilous the journey was for the first pioneers, and the point was driven home by letting the player customize the family members, adding some sentimental stakes to the challenge of getting everyone to Oregon alive. Of course, anyone who played Oregon Trail as a child probably also remembers that the most entertaining part of the game was naming the family members after your siblings and then giggling as they died of dysentery, but it highlights one of the reasons the game was such a success: the player had to make interesting choices, and those choices had real consequences. Oregon Trail had true triangularity because the choice the player made in one event could drastically impact the supplies that were available for the next one. It was challenging to get the entire family to the end, and the player’s choices made each experience unique and interesting. The ability to name the characters after real people added a layer of investment that faceless 78 characters could not necessarily provide. It was common for a player to immediately start up a new game to try again after experiencing failure. Djaouti et al. (2011) say, “Ultimately, [Oregon Trail] clearly shows that an ‘educational’ or ‘serious’ game is not necessarily the opposite to a ‘popular and commercially successful game’” (p. 9). Using the LM-GM framework (Figure 2), Oregon Trail reaches the retention level (the player is immersed in the world of the 1800’s), the understanding level (participation with leading the family on their journey via role-play), and the applying level (progression and collection lead to demonstration and imitation). The analyzing level could be represented in the form of feedback and experimentation, as the player observes what happens in one situation (e.g. the wagon sinks when they attempt to cross) and carries that knowledge into their next playthrough (e.g. their next attempt they might take note of the river depth and attempt to caulk the wagon to float across instead). The game also has elements of evaluation through aspects of resource management as the player attempts to juggle their food, repair equipment, and other supplies within limited inventory space, and rewards/penalties as outcomes for the events and the choices made within them. The player is also encouraged to take ownership of the outcomes of their decisions, bringing in aspects of strategy and planning as they learn what is required to survive the journey to Oregon and to anticipate the challenges, making sure their in-game family is as prepared as possible. The target audience tends to be younger, so it is not guaranteed that they will advance to a creating level where they are experimenting with the outcomes of different decisions. A more advanced gamer might make the jump to using the game events as feedback leading to identifying and analyzing themes, and might even take it further to delve deeper into the information to make more advanced hypotheses about the events they experienced, but it is not necessarily a guaranteed outcome from the game mechanics alone. 79 A way of facilitating the more advanced levels of the LM-GM framework (and therefore Bloom’s Taxonomy) in Oregon Trail might be to have reflection points where the player reflects on the differences between making the journey in the 1800’s vs making the journey in the modern day, or have them attempt to map out a route to a different location and compare and contrast what perils might be encountered (bringing them to the “create” level by designing a game stage for other players), but such design would have to be carefully weighed so as not to burden the pace of the game. Potluck Panic (2016) Potluck Panic was created by the New Mexico State University to help train players in food safety; the game is free to play on their website (New Mexico State University, n.d.). I tested the Unity client version in 2019. The game features nine levels. Each level offers the player two food serving scenarios at a time, and a hand of four cards with various food safety rules on them. The player is tasked with analyzing the scenario and applying the correct card to resolve the safety issue, which is an implementation of a “spot the mistake” instruction strategy that requires the learner to understand the steps involved in order to answer correctly, rather than just memorizing the correct responses. An incorrect match advances the clock one point, while a correct card match will give the player a “safely served” food. Failing to make a correct match before the clock runs out will give the player an “unsafely served” food. The player must serve a number of safe foods before serving a number of unsafe foods in order to complete a level, and the levels have increasingly complex scenarios and more potential card matches. The hand of cards is randomly drawn, and the player can shuffle their hand or use hints. Some cards are special and apply to more than one scenario, or offer bonuses like resetting the clock. The level 80 select gives the player a sense of progression, and the game also featured achievements to provide the player with a sense of completion. In my testing of Potluck Panic, I found that achievements were handed out for almost everything I did, even if it was a minor action, which meant the achievements felt devalued. I also found that it was almost impossible to actually fail a level due to lack of knowledge, meaning the learning experience felt a bit shallow. Time only advanced when a card was played, which helped remove time pressure and stress from the game, but the time reset cards were so plentiful that I often found them choking my hand. Worse, because I had time reset cards in my hand, I often had no room for the card I needed to complete a scenario, meaning I had to deliberately play a wrong card and waste time and then reset it in the hopes of getting the correct card in my newly-freed hand slots. I found this mechanic frustrating because it had nothing to do with my knowledge of the situation—I knew exactly which card to play, but I kept pulling other cards instead. The shuffle mechanic was no help in this scenario because it came with a penalty of time loss, making it no different from playing an incorrect card. I felt I was in a “no win” scenario, forced to randomly draw cards until I got the correct one or received an unsafe food penalty, which was frustrating because I felt I was losing to randomness instead of learning why my choice was incorrect. The frustration made me want to quit playing around level five, but I persisted through all nine levels to see if the game mechanics and challenge changed as the game progressed. The increased levels actually became more frustrating because more cards were added to the game, but there was no way to increase hand size or reduce the shuffle penalty, making it more unlikely that I would have the correct card available. I also found the mechanics simplistic. In many cases you could stop reading the scenario once you recognized the icon for a particular match. There was no need to place the card directly 81 on the matching scenario—simply placing it anywhere on the board would result in a match if one existed. Since the consequence for an incorrect match was the same as the consequence for shuffling, I found myself throwing all of the cards at the board, resulting in random correct matches for scenarios I had not read. Furthermore, a lot of the scenarios were labelled with descriptions like “packed in contaminated ice” or “prepared with unclean hands”. As soon as I saw an obvious “mistake” description, I stopped processing the scenario and started throwing cards at it in hopes of making a match. Since I was not fully processing a scenario, I doubt much meaningful learning was occurring. The inclusion of a hints system was initially appealing, with options like “Ask a scientist”. I thought this would be an interesting way to get information across to the learner and enhance what they were learning from the card matching. Unfortunately, I found the hints almost entirely useless. There were a few situations where a new scenario did not match to the card I expected it would, but the hint system gave me such a vague description that I still had no idea which card to look for afterward. I noticed that some of the hints did not display properly, with information cut off mid-sentence at the bottom of the text box, which makes me wonder if a display error prevented me from receiving the information, and the hints were actually more robust than they appeared. Potluck Panic attempted to include triangularity by offering consequences for wrong actions, but I never actually had any meaningful choices to make. Because I was forced to take penalties as part of the regular gameplay, those penalties could not be severe, making them almost entirely worthless. If I were required to make exact matches, I would have had to put thought into where I placed each card. I feel it would be helpful if the player could upgrade themselves, perhaps earning points for streaks of correct answers that they can spend on hand 82 size or deck modification options, to help reduce the randomness in the final levels. Since the game allowed for combination cards that resolved multiple scenarios, the player could be provided with a way to upgrade existing cards to provide more options within a limited hand. Allowing the player to customize would give them buy-in to the mechanics and get them engaged in doing it correctly, and allowing them to deliberately modify the game would shift the mechanics from pure randomness to a consequence of the player’s progression choices. Instead of being frustrated at pulling the wrong card repeatedly, the player would be more focused on obtaining the correct upgrades to reduce the frequency of pulling the wrong card. Using the LM-GM model (Figure 2), Potluck Panic reaches the retention level (the player is exposed to a story and the basic facts of food safety), but any further progress on Bloom’s hierarchy depends on whether the player was making matches with care. The player can drop cards on the board with no penalty and make matches at random, which means they do not truly reach the level of understanding (answering questions) or applying (demonstration or simulation). Therefore, the game could be a useful tool for advancing to the levels of analyzing and evaluating, and potentially even all the way to creating (if the player was presented with a level where they had to create a food safety scenario on their own), but the current design really leaves it up to the user to want to learn, rather than presenting the information in a way that facilitates learning via play. Malaria Invasion (2019) Malaria Invasion is a game produced by Drexel University to teach learners about the mechanics of malaria infections (Drexel University, n.d.). I tested the Android version of the app from Google Play. While the app itself did not have a visible version number, I first tested it in 2019 and when I returned to it in 2023 I noticed the game had been updated to include more 83 elements, addressing a few of my initial observations. These updates suggest that this app may still be under active development. Malaria Invasion puts the player in the role of a malaria virus attempting to infect a host. In the first level, the player infiltrates the host by tagging red blood cells in a vein, while dodging the white blood cells that seek to destroy them. In the second level, the player attempts to attach the virus to the red blood cell. In the third level, the player attempts to infiltrate the blood cell. The first thing I noticed about Malaria Invasion was that it was bombarding me with technical terms with no information about them. Perhaps the terminology is more appropriate for their target audience of graduate students and research trainees in infectious disease (Drexel University, n.d.), but, despite my biology background, I found a lot of the terms faded into “noise”. There was a button that let the player access scientific facts and explanations, but it was completely disconnected from the gameplay itself which meant the player would have to deliberately seek the information to be exposed to it. The game also did not pause, which meant accessing the information was potentially detrimental to gameplay as it led to missing targets while reading. Placing the information behind a disconnected button is a direct contradiction to the typical goal of a serious/educational game, which is to convey knowledge through gameplay rather than through a user manual (Arnab et al., 2015). Given enough exposure, I could see the terminology sinking in and becoming more familiar, which could aid in learning in real-world scenarios; but there were only three stages, and each took a minute or less to complete. Worse, there was no replay value, which meant I had no reason to be further exposed to the information. Malaria Invasion held absolutely no stakes for me as a player. The game showed me how to infect a cell, but it never actually explained to me why I would want to do that. As a researcher, the player might recognize that it is beneficial for a virus to infect a cell because it 84 allows for spread, but as a player, I was dumped into the scenario and given a goal with no real motivation to complete it; especially given the negative connotations associated with viruses. The game even depicted the virus in cartoon form with a scowling face, cementing its villain role. Expecting a player to want to help a virus succeed at infection is a lot like expecting them to work against their favourite storybook heroes. The narrative needed to be expanded to allow me to assume my role as a virus and become motivated to complete it. In the first stage, the virus rushed through a vein and the player had to control it to touch red blood cells without touching white blood cells. The stage had a timer to represent when the immune system would detect the infection, so the stage was a race to collect as many red blood cells as possible before being detected. Touching a white blood cell would reduce the timer and alert the immune system to the infection, reducing the amount of time available to collect red blood cells, which is a good implementation of some consequences for actions in the game, but it was difficult to link the actions to the overall goal of the game because the player progressed to the next stage regardless of how many cells they touched. It was also not immediately apparent why the virus would need to collect so many blood cells, since the next stages of the game only involved one cell each. In the second stage, I tried to shoot “bullets”, that represented proteins, into targets that represented protein receptors, with an ultimate goal of filling the virus’s calcium reserves. There was no consequence for missing a target, and the only real decision I made during this stage was the timing on when to fire the protein to line it up with the moving targets. I could not decipher what the difference was between the different receptors or which ones to prioritize, so I fired the proteins randomly. It was difficult to understand why I needed to fill the calcium reserves 85 without reading through the optional information screens, which suggests that the information was not easily transferred as a natural part of the gameplay. The third stage required me to shoot more targets to line up the virus to complete infiltration, and the only difference was that the aiming was from a different angle than the second stage. There were still no consequences for missing the targets, other than taking more time to complete the stage. In both the second and third stages, the lack of consequences meant the stage had no real challenge, and I could rapidly tap the button to pass each level without absorbing much of the information. After I completed the three levels, the game congratulated me and presented a list of learning outcomes. The first learning outcome said, “You have learned the importance of the immune system”, referring to the white blood cells in the first stage. The game also announced to me that I had “learned how to attach to a red blood cell using your microneme and rhoptry adhesion proteins”. I had not really connected those terms to the “bullets” in stage two, which suggests the terminology was not learned at all. Then the game informed me that I had “learned the importance of increasing intracellular calcium stores”. Even after reading the description on the webpage, I am still not sure what the calcium stores do, or whether it was good for the virus (since I was playing as the virus) or the host (since the first learning outcome emphasized the importance of the immune system for the host, not the virus). The final learning outcome said, “You have learned the use of an actin-myosin motor to propel a moving junction and complete your invasion”. The outcome added a bit more context to stage three, but I am not sure that any of these learning outcomes actually ‘stick’ without more immersion. The 2023 version of the app has revised these learning outcomes to read as “This adventure has shown you” which is more accurate and removes the problematic assumptions. 86 I would consider the Malaria Invasion app as more of a supplement to a lecture or course, and more of a simulator than a full game. A student who had already learned most of these terms and actions could probably gain something from playing through it, but a player without that external context could easily be confused. If the game had more stakes, getting the player interested in playing for fun rather than just as a demonstration, the immersion in (and eventual familiarity with) the terminology and mechanisms could potentially be transferred to real-world studies. As it was, there were no interesting decisions to be made and no real reason to play other than to see the mechanisms at work. If the game design had introduced me to the role of virus in a more compelling way and gotten me invested into the role as a player, I would have been more interested in performing a successful infection on a cell. The player could begin as a sole virus recruiting red blood cells to its cause, eventually expanding into a full-blown infection as the ranks of infected cells grow. The game could be quite effective at demonstrating the role of the immune system at each stage of infection, as the player suffers real consequences for failing to evade it. The culmination of the game could demonstrate the spread of the virus to a new host, starting the infection cycle over again. As it was, the game took almost no time to play and left me with little reason to play again. Using the LM-GM model (Figure 2), Malaria Invasion reaches the retention level (by presenting the information, although it needed more of a narrative to really immerse the player), and it attempts to reach the understanding level through role-play and participation, but it is debatable whether it is successful. I am not sure that I, as a player with no background in these specific viral mechanisms, could have successfully answered questions after playing the game. A player who was already exposed to this information and was using the app as a demonstration/simulation would probably find it easier to advance to ‘understanding’ and 87 onward, but the act of playing the game alone did not achieve that for me. If the player is able to move past that point, Malaria Invasion potentially reaches the “applying” level through movement and time pressure (at least for the first stage) as well as imitation and simulation, and the analysing level through realism and shadowing (although more as a simulation than as a game). However, since there are no real rewards or resource management and there is no opportunity to reflect on the information within the game, I would not consider Malaria Invasion to have reached the evaluating level. The app would need to explain more fully what was happening, and allow the player to experience what happens when those mechanisms fail, to foster a better understanding of what is and what is not happening. If the gameplay was expanded to allow the player to manage a whole battalion of viruses, recruiting cells and trying to infect new hosts, there would be a better sense of progression (moving to the applying level), and the player would have a better understanding of how the mechanisms interact (moving to the analyzing level). If designed well, the player could also experiment with different infection vectors (evaluating level) and possibly even attempt to reengineer the virus to test different infection strategies (moving to the create level). Academic Integrity in Cyberspace (2023) A partnership between Seneca College and Toronto Metropolitan University in collaboration with eCampus Ontario and funding from the Government of Ontario helped to produce “Academic Integrity in Cyberspace (AIC)”, a gamified web module designed to help promote academic integrity (Leithead et al., 2023). I tested the HTML version of this game in August 2023, and I provided feedback to the developers, which they enthusiastically accepted and said they would consider for the next round of updates. 88 In AIC, the player assumes the role of a student within the fictional Akadema Galaxy and helps to defend their classmates from the evil Captain Corruptus. The game begins by introducing the player to Captain Corruptus’s sinister plan to weaken students with his Misinformation Ray, tricking them into quitting school and joining his “League of the Unearned”, in order to build an army to overthrow the Akadema Galaxy. The player is presented with a choice between three planets, which correspond to themed modules (e.g.: “collaboration” or “use of sources”). Upon selecting a planet, the player is presented with a scenario where their classmate either asks for advice or suggests a course of action, and the player must select the response that results in acting with academic integrity. For example, a classmate might reveal that they have discovered a free paper online that will save them time on their assignment, and the player can choose between encouraging them and asking for access to the paper as well, or pointing out that using someone else’s work is an example of misconduct. After selecting their response, the player is provided feedback and more information about the choices, and awarded a point on the scoreboard for either the player or Captain Corruptus. Accumulating too many points for Captain Corruptus will result in a quiz where the player has to battle Captain Corruptus directly by answering additional questions about the module theme. Failing to defeat Captain Corruptus will result in the planet icon receiving the Corruptus logo, and the player will need to replay the scenarios to reclaim the planet. I found that the player had no real opportunity to make meaningful choices within the game. The player was presented with a choice of which planet to visit, and every scenario after that was a linear track where one situation would end and the next would begin with no choice of who to talk to next. If the player responded incorrectly to a scenario, the Captain Corruptus team would accumulate points, but the choices would not affect progression to the next scenario or 89 have any effect on how the next scenario is presented. Progression through the game was in the form of completing the planets, with the scoreboard indicating how many scenarios were answered with academic integrity, but the player does not accumulate anything meaningful other than progression to the next scenario. In my opinion, most players would not consider achieving “Slide 8 of 16” to be gameplay or meaningful progression in a game, and the progression in AIC felt somewhat equivalent. I also found that every scenario was a binary option of either agreeing or disagreeing with the NPC, meaning the responses were limited and had no subtlety. Furthermore, the correct response in every scenario was to disagree with the NPC. Not one fictional student in the Akadema Galaxy is making a correct decision, which is not only sad, but also meant it was trivial to select the correct response without putting any analysis into it. If AIC is examined through the LM-GM model (Figure 2), it easily reaches the retention level, because the player is exploring the planets and receives feedback and guidance based on their responses to the scenarios. It reaches the understanding level by offering roleplay and question and answer options. It reaches the applying level by allowing the player to demonstrate their knowledge and through the simulation of potential scenarios, but there is no opportunity to move around the map (you are locked into a planet on a linear path after making the initial choice), there is no time pressure, and there are no consequences (failing a module reveals an option to replay to receive a different result. The scenarios within will remain the same). Even though each response results in feedback, the player has no real option to act on that feedback and there is no opportunity to experiment (beyond deliberately selecting the wrong answer to see what will happen) or apply what is learned through that feedback in the game itself. The lack of freedom to experiment and the limited structure of the scenarios (the NPC is always incorrect) means that, in my opinion, AIC does not achieve the analysing or evaluating levels of the LM- 90 GM model. There is no opportunity or need to plan actions or strategize, and there are no aspects of resource management or optimization. There is no need for the player to take ownership or responsibility for their actions in the game because there are no real consequences for an incorrect answer (the consequence for completely failing and allowing Captain Corruptus to win a planet is that the player has an option to play it again. None of the scenarios within a planet will be affected by the previous choices). Therefore, I do not believe AIC reaches the creating level of the LM-GM. To make a truly immersive roleplaying experience that reaches the creating level of the LM-GM, the AIC game could have more of an “open world” style where the player is free to explore the planet and their school and choose to talk to their classmates. Rather than having one scenario lead linearly to the next, the player could have the option of exploring, chatting with the NPCs and learning what they are struggling with, and then possibly even chatting with other nearby NPCs to ask for opinions in order to get hints on how to resolve the scenarios. The scenarios presented addressed important situations and could be thought-provoking, but the NPCs would need to have to have varying responses (perhaps even multiple NPCs that are presenting differing points of view to consider) that made it necessary for the player to put some analysis into the situation in order to determine the correct course of action. To truly reach the creating level, the game would ideally have some sort of resource management or time pressure, so the player could potentially be dealing with an inventory of items or have a need to recruit the NPCs in order to resolve some of the more difficult scenarios, giving the player a greater purpose in exploring to discover items or allies. In the current game, the player never gets a chance to resolve the Captain Corruptus situation, so a more robust narrative where the player accumulates 91 resources and allies in order to banish the bad guys could result in more rewarding and compelling gameplay. The State of Post-Secondary Educational Games Having identified triangularity as a mechanic that introduces rewards/penalties and potential resource management (the evaluating level of the LM-GM), and strategy/planning which leads to ownership of the consequences of the player’s choices (the creating level of the LM-GM), I conducted this survey of currently-available educational games for post-secondary looking for both evidence of triangularity and also evidence of a game that reached the top level of LM-GM (and therefore the top level of Bloom’s Taxonomy). I was unsuccessful at discovering games that use these mechanics. It is unclear whether triangularity is neglected in educational game design because it is not well-known as a mechanic, or if it is because the design of the game itself becomes much more complicated and difficult to produce when attempting to include triangularity-style elements. If triangularity is a mechanic that could potentially facilitate the student/player reaching the top level of the LM-GM or Bloom’s Taxonomy, it may be worth putting in the extra effort to design a more complicated (and potentially more expensive and time-consuming to develop) game that includes triangularity elements. If the inclusion of triangularity also makes the game more compelling for the player and helps to dispel the “boring” stigma of educational games, leading to more play time and therefore more interaction with the included material, it may justify the development investment into more complicated design and mechanics. However, it should be cautioned that injecting triangularity is likely insufficient to make a game reach the creating level. Even if the creating level is reached, it is important to ensure that the player is “creating” with the learning material, not just the game mechanics, otherwise the 92 player has learned the mechanics instead of the intended material. Careful consideration must occur as the game is designed to ensure the focus is not misplaced. But as long as that consideration is incorporated, triangularity could be a tool that helps the game, and player, move closer to the creating level and content mastery. Design of the Game in this Study The design of my game, in both triangularity and non-triangularity form, reaches the evaluating level in the LM-GM by introducing elements of resource management (determining how much time is necessary and working to accumulate it). Both forms of the game include accountability and responsibility in the form of the opportunity to recognize when the challenges outstrip understanding (ideally leading to increased efforts to improve in those areas), and strategy and planning with being able to gauge how much time would be require to complete the final engine question, which is equivalent to writing an exam in the course. Both versions of the game also contain aspects of the creating level element of infinite game play in that the player can continue to practice each of the chapters and fill the status bars to light up the completion checkmarks as much as desired, but as previously noted, the “infinite gameplay” link to creating felt a bit weak. The triangularity form of the game includes additional strategy and planning elements (determining whether to take the safer path or the more risky but more lucrative path to get to their goal amount of time before attempting the final question), and the accountability of understanding the material before going into a series of questions (or risking losing earned time). The game falls a bit short of a more complete creating level in that the player has no real opportunity to manipulate the content and use the materials in novel ways within the game framework, nor is there a multiplayer element to facilitate a protégé effect, but this could be 93 included in a re-design of the same game, provided the resources were available to facilitate development. A more robust creating level design would allow the player to receive feedback specific to any mistake they made on a question, and allow them to select another question that allowed them to experiment with the specific concept that they struggled with (manipulating the content mechanics directly and allowing them to “create” the question they are practicing on, rather than having a randomly assigned question from a chapter). A proper “creating level” protégé effect could have been facilitated if the game were cooperative multiplayer or had teams working together to solve the problems. Allowing the player to visually rebuild the ship, choosing where to place parts as they repair systems, ultimately coming up with a design of their own creation and engaging them with the content as they strive to find the parts they need to complete their ship, could be another possibility. These more advanced designs were not possible with the resources available for the development of this version of the game.