Systematic literature review update

The November 2015 issue of Computers and Education contains the paper:

An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games.

Authors: Elizabeth Boyle, Thomas Connolly, Thomas Hainey, Grant Gray, Jeffrey Earp, Michela Ott, Theodore Lim, Manuel Ninaus, Claudia Ribeiro and Joao Pereira.

Data examined: 143 papers from 2009-14.

Abstract: Continuing interest in digital games indicated that it would be useful to update Connolly et al.’s (2012) systematic literature review of empirical evidence about the positive impacts and outcomes of games. Since a large number of papers was identified in the period from 2009 to 2014, the current review focused on 143 papers that provided higher quality evidence about the positive outcomes of games. Connolly et al.’s multidimensional analysis of games and their outcomes provided a useful framework for organising the varied research in this area. The most frequently occurring outcome reported for games for learning was knowledge acquisition, while entertainment games addressed a broader range of affective, behaviour change, perceptual and cognitive and physiological outcomes. Games for learning were found across varied topics with STEM subjects and health the most popular. Future research on digital games would benefit from a systematic programme of experimental work, examining in detail which game features are most effective in promoting engagement and supporting learning.

My notes: This is a comprehensive, but also easy, read for those of us interested in the evidence or proof for the effective use of digital games in teaching and learning. The data is clearly presented and discussed, and the listing of coded papers that closes the paper is a mine of relevant literature.

More information at:

Secret Sartre

Games about human life are often interesting, and for those in academia or who have spent time in there, games about university life specifically can hold a special fascination. From Scandinavia, here’s the introduction to a card-based game (complete with the card designs). Secret Sartre:

In Secret Sartre, the faculty members of an unnamed university department battle for ideological supremacy. A fragile alliance of upstanding rationalists, logical positivists, empiricists, liberal humanists, scientists and other fetishizers of the Enlightenment must work together to stem the rising tide of postmodernism. Watch out, though – there are closet postmodernists among you, and someone is Secret Sartre.

At the beginning of the game, each player is secretly assigned to one of three roles: Science, Postmodernism, or Sartre. Sartre plays for the postmodern team, and the postmodernists know who Sartre is, but most of the time Sartre does not know who his fellow postmodernists are. The Scientists are far out on the autism spectrum and don’t know who anyone is.

The scientists win by enacting five rational policies or having Sartre fired. The postmodernists win by enacting six postmodernist policies, or if Sartre is elected Head of Studies late in the game. As postmodernist policies are enacted, the Department Chair gains new powers. Even scientists may find themselves tempted to enact postmodernist policies that help them control the table and assassinate their enemies.

From the rules, how the cards appear:

Secret Sartre cards

This isn’t a quick game to pick up; the rules need a bit of figuring out, and a familiarity with card games, and academic research philosophy, helps. You also need several players to get the full benefit of Secret Sartre. But even if you don’t play, the description of the rules is amusing, and it’s an interesting way of comparing and contrasting academic and research positions.

Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis

In the April 2015 edition of Review of Educational Research can be found:

Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis.

Authors: Douglas B. Clark, Emily E. Tanner-Smith, and Stephen S. Killingsworth.

Abstract: In this meta-analysis, we systematically reviewed research on digital games and learning for K–16 students. We synthesized comparisons of game versus nongame conditions (i.e., media comparisons) and comparisons of augmented games versus standard game designs (i.e., value-added comparisons). We used random-effects meta-regression models with robust variance estimates to summarize overall effects and explore potential moderator effects. Results from media comparisons indicated that digital games significantly enhanced student learning relative to nongame conditions (g = 0.33, 95% confidence interval [0.19, 0.48], k = 57, n = 209). Results from value added comparisons indicated significant learning benefits associated with augmented game designs (g = 0.34, 95% confidence interval [0.17, 0.51], k = 20, n = 40). Moderator analyses demonstrated that effects varied across various game mechanics characteristics, visual and narrative characteristics, and research quality characteristics. Taken together, the results highlight the affordances of games for learning as well as the key role of design beyond medium.

My notes: One swallow doesn’t make a summer, and one paper doesn’t “prove” that digital games are jolly useful things to use in education, learning and teaching. However, every so often an article, paper or report of the thousands (yes, thousands) published on games in learning every year comes along that does show something significant, has some persuasive analysis in it, and is definitely worth a read. This recent paper is one. The work looks at research published between 2000 and 2012 and was funded by the Bill and Melinda Gates Foundation. There’s a brief, and far less technical, summary document which introduces the various hypotheses. It’s a long text; the statistics within are somewhat hardcore (and my first degree was in statistics), and it’s a good few hours of concentrated reading. The reference section is also pretty good.

More information at:

n.b. Thanks to Doug for a copy of a version of the paper.

The effectiveness of instructional games: a literature review and discussion.

A 2005 report from the US Naval Air Warfare Center Training Systems Division:

The effectiveness of instructional games: a literature review and discussion.

Author: Robert T. Hays.

Abstract: This report documents a review of 48 empirical research articles on the effectiveness of instructional games. It also includes summaries of 26 other review articles and 31 theoretical articles on instructional gaming. Based on this review the following 5 conclusions and 4 recommendations are provided. Conclusions: (1) The empirical research on the instructional effectiveness of games is fragmented, filled with ill defined terms, and plagued with methodological flaws. (2) Some games provide effective instruction for some tasks some of the time, but these results may not be generalizable to other games or instructional programs. (3) No evidence indicates that games are the preferred instructional method in all situations. (4) Instructional games are more effective if they are embedded in instructional programs that include debriefing and feedback. (5) Instructional support during play increases the effectiveness of instructional games. Recommendations: (1) The decision to use a game for instruction should be based on a detailed analysis of learning requirements and tradeoffs among alternate instructional approaches. (2) Program managers and procurement officials should insist that instructional game developers demonstrate how their game will support instructional objectives. (3) Games should be used as adjuncts and aids, not as stand-alone instruction. (4) Instructor-less approaches (e.g., web-based instruction) must include all “instructor functions.”

My notes: Though this is from 2005, from the military as opposed to the education sector, and is specifically focused on instructional games, there is much of interest and contemporary relevance in this literature review. It’s a long read, and written more plainly than comparable academic texts (though this aids the speed of reading). The author took a rigorous approach to weeding out candidate documents that may not be suitable for examination. Pages 37-41, “other issues in instructional gaming”, summarise a collection of interesting issues including adding instructional support to a game, learning using an adventure game, evaluator interest (and bias), the instructional value of a game developed for another purpose, and the importance of debriefing.

More information:

Systematic literature review

The September 2012 issue of Computers and Education contains the paper:

A systemic literature review of empirical evidence on computer games and serious games.

Authors: Thomas Connolly, Elizabeth Boyle, Ewan MacArthur, Thomas Hainey and James Boyle.

Abstract: This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.

My notes: This is an excellent review of nearly 130 papers, from 2004 to early 2009, concerned with the use of computer and serious games in learning. It’s an easy read, but has much substance, presented in a neutral manner.

More information at: