stage 3: Designing with the ATMSG
Game Frameworks & Why the ATMSG
Many game frameworks have been developed to evaluate and design serious games on their quality--primarily judging how well the gaming components/activities support the proposed learning outcomes of the game (i.e. conceptual integration). I chose to use the Activity Theory Model of Serious Games (ATMSG; Caravalho et al., 2015) due to its thoroughness and its evaluation on a games conceptual integration at a multitude of levels compared to other frameworks.
While a small caveat to this framework (and many existing frameworks for serious games) is that it is an entirely qualitative approach, I employed an extended version of the ATMSG proposed by Callaghan, McShane, Eguíluz, & Savin-Baden that adds a game-trace layer (see below) that accounts for elements within the game to be recordable such that they can be statistically analyzed. Below are excerpts from my design documentation that shows the ATMSG applied to Sugar Scramble.
Step 1: Describing Game Activities
According to the ATMSG, a serious game is comprised of activities: the gaming activity, learning activity, and intrinsic instruction.
Step 2: Mapping out a Game Sequence
The sequence of gaming events and components are then organized in a map based on the descriptions of the gaming activity in Step 1 to communicate the overall flow of a player’s experience with the game.
Step 3: Generally Describing Game Mechanics
In this step of the ATMSG framework, each of the mechanics described in the game sequence in Step 2 are briefly described using “buzzwords” or short phrases. This is helpful to see if any of the gaming activities, learning activities, or intrinsic instruction conflict with each other or if any components do not actually support their respective activities.
Step 4: Generally Describing Game Mechanics
This step is the addition to the original ATMSG which allows for quantitative analysis of the serious game given that there is back-end development for the game. Below, data points (game components) are identified and are described as to what they convey when they occur in-game. Due to the specific interest in evaluating the effectiveness of teaching the opportunities of productive negativity that serious games provide, these data points specifically relate to instances of failure (i.e. when there are gaps in a player’s knowledge, and what that gap is in particular) and assessing change in player performance.
Data points 1., 2., and 3. are instances that the player receives negative feedback due to a gap in their knowledge. 1. indicates that the player has a misconception related to what the substrates are for a particular enzyme; 2. indicates that they have a misconception related to the inter-pathway connections/relationships for that biological context; and 3. indicates the player has a misconception related to where the enzyme is located in the cell. During an evaluation, these instances of failure can be recorded and compared within the same level or between different levels (the latter is possible due to the fact that many of the levels build upon each other and share some game components/goals. Ideally, if the player is showing that they are learning, the number of these errors will decrease as the player replays a level or plays more levels.
Productive negativity can be assessed more granularly using data points 1., 2., and 3. as described in the previous subsection on assessing failure, but it can also be assessed more broadly by recording the number of times a level is replayed until it is completed perfectly (i.e. with a 3-star score). A 1- or 2-star score indicates that the player has failed or partially failed in some way; if a player is able to take this feedback and eliminate the number of errors such that they complete the level perfectly, that is an indication of productive negativity.
Recording the number of times the help button is pressed can provide insight into why the player may be failing a level or making mistakes. Since the help section of the game provides instruction on how to play the game and is not related to any biochemistry content, a player frequently accessing this button may indicate that failure is a result of the game being too difficult rather than an indication of the player’s lack of knowledge.
Step 5: Describing Game Mechanic Implementation in Detail
The final step of the ATMSG is to group each activity’s set of actions, tools, and goals for each mechanic, and provide a detailed description of the implementation of the “buzzwords” listed in Step 3 (i.e. explain what is being done in the game using what tools and to what purpose).
References
-
Callaghan, M., Mcshane, N., Eguíluz, A. G., & Savin-Baden, M. (2018). Extending the activity theory based model for serious games design in engineering to integrate analytics. International Journal of Engineering Pedagogy (iJEP), 8(1), 109. doi:10.3991/ijep.v8i1.8087
-
Carvalho, M. B., Bellotti, F., Berta, R., De Gloria, A., Sedano, C. I., Hauge, J. B., Rauterberg, M. (2015). An activity theory-based model for serious games analysis and conceptual design. Computers and Education, 87, 166–181