Category Archives: Evidence

High quality literature reviews and analysis of studies concerning the use of games in learning.

Systematic literature review update

The November 2015 issue of Computers and Education contains the paper:

An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games.

Authors: Elizabeth Boyle, Thomas Connolly, Thomas Hainey, Grant Gray, Jeffrey Earp, Michela Ott, Theodore Lim, Manuel Ninaus, Claudia Ribeiro and Joao Pereira.

Data examined: 143 papers from 2009-14.

Abstract: Continuing interest in digital games indicated that it would be useful to update Connolly et al.’s (2012) systematic literature review of empirical evidence about the positive impacts and outcomes of games. Since a large number of papers was identified in the period from 2009 to 2014, the current review focused on 143 papers that provided higher quality evidence about the positive outcomes of games. Connolly et al.’s multidimensional analysis of games and their outcomes provided a useful framework for organising the varied research in this area. The most frequently occurring outcome reported for games for learning was knowledge acquisition, while entertainment games addressed a broader range of affective, behaviour change, perceptual and cognitive and physiological outcomes. Games for learning were found across varied topics with STEM subjects and health the most popular. Future research on digital games would benefit from a systematic programme of experimental work, examining in detail which game features are most effective in promoting engagement and supporting learning.

My notes: This is a comprehensive, but also easy, read for those of us interested in the evidence or proof for the effective use of digital games in teaching and learning. The data is clearly presented and discussed, and the listing of coded papers that closes the paper is a mine of relevant literature.

More information at:

Game-based learning: latest evidence and future directions

In 2013, the National Foundation for Educational Research released this report:

Game-based learning: latest evidence and future directions.

Authors: Carlo Perrotta, Gill Featherstone, Helen Aston and Emily Houghton.

Data examined: 31 works of various types from 2006-13.

Abstract: This review is the first output in the Innovation in Education strand of NFER’s research programme. This strand will provide evidence about new approaches to education, teaching and learning and aims to identify rewarding learning experiences that will inspire, challenge and engage all young people, equipping them with the essential skills and attitudes for life, learning and work in the 21st Century. Interest around the use of video games in education is high, and following the emergence of new trends like ‘gamification’, Futurelab@NFER felt that it was timely to provide educators, industry and researchers with an up-to-date analysis of the literature.

To achieve this, we conducted a rapid review of the latest available evidence, seeking to answer these research questions:

  • What is game-based learning?
  • What is the impact and potential impact of game-based learning on learners’ engagement and attainment?
  • What is the nature and extent of the evidence base?
  • What are the implications for schools?

The research questions are mainly concerned with the notion of ‘gameplay’ (playing games) rather than ‘making games’ (how the prospect of creating original video games can be used to interest young people in complex activities like software programming).

Excerpt: Despite some promising results, the current literature does not evidence adequately the presumed link between motivation, attitudes to learning and learning outcomes. Overall, the strength of the evidence has been affected by the research design or lack of information about the research design.

My notes: This recent work is a less academically-analytical work, and more wider in remit, than a typical meta-review of evidence. The content and format are, in addition, designed for a more non-academic readership. However, it does not suffer because of this and there is much here of interest to both academics and practitioners. The appendices pleasingly includes thorough details of the search strategy, review process and the evidence base for the review.

More information at:

Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis

In the April 2015 edition of Review of Educational Research can be found:

Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis.

Authors: Douglas B. Clark, Emily E. Tanner-Smith, and Stephen S. Killingsworth.

Abstract: In this meta-analysis, we systematically reviewed research on digital games and learning for K–16 students. We synthesized comparisons of game versus nongame conditions (i.e., media comparisons) and comparisons of augmented games versus standard game designs (i.e., value-added comparisons). We used random-effects meta-regression models with robust variance estimates to summarize overall effects and explore potential moderator effects. Results from media comparisons indicated that digital games significantly enhanced student learning relative to nongame conditions (g = 0.33, 95% confidence interval [0.19, 0.48], k = 57, n = 209). Results from value added comparisons indicated significant learning benefits associated with augmented game designs (g = 0.34, 95% confidence interval [0.17, 0.51], k = 20, n = 40). Moderator analyses demonstrated that effects varied across various game mechanics characteristics, visual and narrative characteristics, and research quality characteristics. Taken together, the results highlight the affordances of games for learning as well as the key role of design beyond medium.

My notes: One swallow doesn’t make a summer, and one paper doesn’t “prove” that digital games are jolly useful things to use in education, learning and teaching. However, every so often an article, paper or report of the thousands (yes, thousands) published on games in learning every year comes along that does show something significant, has some persuasive analysis in it, and is definitely worth a read. This recent paper is one. The work looks at research published between 2000 and 2012 and was funded by the Bill and Melinda Gates Foundation. There’s a brief, and far less technical, summary document which introduces the various hypotheses. It’s a long text; the statistics within are somewhat hardcore (and my first degree was in statistics), and it’s a good few hours of concentrated reading. The reference section is also pretty good.

More information at:

n.b. Thanks to Doug for a copy of a version of the paper.

The effectiveness of instructional games: a literature review and discussion.

A 2005 report from the US Naval Air Warfare Center Training Systems Division:

The effectiveness of instructional games: a literature review and discussion.

Author: Robert T. Hays.

Abstract: This report documents a review of 48 empirical research articles on the effectiveness of instructional games. It also includes summaries of 26 other review articles and 31 theoretical articles on instructional gaming. Based on this review the following 5 conclusions and 4 recommendations are provided. Conclusions: (1) The empirical research on the instructional effectiveness of games is fragmented, filled with ill defined terms, and plagued with methodological flaws. (2) Some games provide effective instruction for some tasks some of the time, but these results may not be generalizable to other games or instructional programs. (3) No evidence indicates that games are the preferred instructional method in all situations. (4) Instructional games are more effective if they are embedded in instructional programs that include debriefing and feedback. (5) Instructional support during play increases the effectiveness of instructional games. Recommendations: (1) The decision to use a game for instruction should be based on a detailed analysis of learning requirements and tradeoffs among alternate instructional approaches. (2) Program managers and procurement officials should insist that instructional game developers demonstrate how their game will support instructional objectives. (3) Games should be used as adjuncts and aids, not as stand-alone instruction. (4) Instructor-less approaches (e.g., web-based instruction) must include all “instructor functions.”

My notes: Though this is from 2005, from the military as opposed to the education sector, and is specifically focused on instructional games, there is much of interest and contemporary relevance in this literature review. It’s a long read, and written more plainly than comparable academic texts (though this aids the speed of reading). The author took a rigorous approach to weeding out candidate documents that may not be suitable for examination. Pages 37-41, “other issues in instructional gaming”, summarise a collection of interesting issues including adding instructional support to a game, learning using an adventure game, evaluator interest (and bias), the instructional value of a game developed for another purpose, and the importance of debriefing.

More information:

Systematic literature review

The September 2012 issue of Computers and Education contains the paper:

A systemic literature review of empirical evidence on computer games and serious games.

Authors: Thomas Connolly, Elizabeth Boyle, Ewan MacArthur, Thomas Hainey and James Boyle.

Abstract: This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.

My notes: This is an excellent review of nearly 130 papers, from 2004 to early 2009, concerned with the use of computer and serious games in learning. It’s an easy read, but has much substance, presented in a neutral manner.

More information at: