As the use of technology in nursing education is quickly expanding, the science of virtual simulation is just beginning to emerge. The literature reveals a range of products, potential uses, and pilot studies related to virtual simulation; however, the outcomes of the intervention of virtual simulation as well as best practices for its use are yet to be thoroughly studied or established. In addition, faculty and researchers demonstrate a lack of a clear understanding of the definition of virtual simulation. No single terminology is used consistently within the literature of games, serious games, virtual worlds, virtual patients, and virtual reality.1,2 A clear definition of virtual simulation is needed to remove ambiguity, better guide education efforts, and advance the science of simulation.
In 2013, Lopreiato3 led a working group to create the Healthcare Simulation Dictionary published by the Agency for Healthcare Research and Quality (AHRQ). The goal of this landmark document was to enhance communication and clarity for health care simulationists in teaching, education, assessment, research, and systems integration activities.3 This reputable resource has served as a key reference for those seeking understanding of simulation terminology. In this dictionary, virtual simulation is currently defined as
- The recreation of reality depicted on a computer screen (McGovern, 1994).
- A simulation involving real people operating simulated systems. Virtual simulations may include surgical simulators that are used for on-screen procedural training and are usually integrated with haptic device(s) (McGovern, 1994; Robles-De La Torre, 2011).
- A type of simulation that injects humans in a central role by exercising motor control skills (for example, flying an airplane), decisions skills (committing fire control resources to action), or communication skills (as members of an air traffic control team) (Hancock et al, 2008).4
This definition was a sound starting point, but as the science has expanded and evolved, so has the need to re-examine terminology to reveal society's interpretation.
Although the use and understanding of the “computer sense” of virtual simulation has been documented as early as 1959,3 there is limited synthesized knowledge about the learning outcomes that result from virtual simulation as a pedagogy in nursing. Therefore, a systematic review was undertaken to examine, appraise, and synthesize the research about virtual simulation in nursing education to reveal the actualized student learning outcomes. This review used Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) 2009 Guidelines.5
OBJECTIVE
The objective of this review was to identify how virtual simulation impacts nursing student learning outcomes.
METHODS
Study Question
The guiding question for the review was, “In nursing students, how does virtual simulation impact learning outcomes?”
Study Eligibility
Broad inclusion criteria were delineated with the intent of casting the net widely, so to speak, to capture how virtual simulation has been used in nursing education to gather knowledge about the learning outcomes. Inclusion criteria included the following: (a) peer-reviewed, (b) research study of any kind (all study designs included), (c) written in the English language, (d) involving virtual simulation, (e) with nursing students, and (f) from 1996 to 2018. The intervention examined was use of virtual simulation. Foronda's (2018) definition of virtual simulation was applied: “clinical simulation offered on a computer, the Internet, or in a digital learning environment including single or multiuser platforms.”6 This definition was applied because it was current and had been used in the context of nursing education. Exclusion criteria included the following: (a) articles that were not primary research studies, (b) dissertations, (c) language besides English, (d) did not involve virtual simulation, (e) participants that did not include nursing students, and (f) articles that focused on instrument development.
Study Identification
A team of 5 researchers sought the assistance of a library scientist to guide the search process. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed.5 Data sources included articles retrieved from the databases of PubMed, CINAHL, Web of Science, and ProQuest. Databases were searched using MeSH terms and a key word search of “virtual simulation,” “education,” and “nursing,” combined with the Boolean operator of “AND.” Additional search limits were applied including publication date range of 1996 to 2018 to be consistent across databases as well as limits to include scholarly, peer-reviewed journals or research studies.
Study Selection
Two researchers worked independently to screen article titles and abstracts. When there was a discrepancy or uncertainty, a third researcher reviewed the title and abstract and helped establish consensus. The interrater agreement on the initial screening was calculated to be 75%. Only 50 relevant research studies were selected. Next, an ancestral approach, meaning a search of the reference lists for additional potentially relevant articles based on titles, was then conducted. A second ancestral approach of the newly identified articles was conducted. These 2 searches identified an additional 163 articles. Duplicates were then removed. In total, 80 research studies were included in the review (Fig. 1).
;)
FIGURE 1:
Flow chart.
Data Extraction
Studies were read, and data were placed into summary tables to assist with systematic documentation of the appraisal and synthesis of findings. The summary tables included data reflecting various characteristics of the individual studies, ranking and appraisal of the studies, and categorization of the research findings. Two researchers independently ranked the articles on a hierarchy of 1 (high-quality evidence) to 7 (low-quality evidence) using the Melnyk and Fineout-Overholt's (2015) levels of evidence.7 Next, the researchers independently appraised each study for bias using the Critical Appraisal Skills Programme (CASP) guidelines.
The Jeffries (2005) Simulation Framework was chosen as the framework for analysis of data.8 Although the model has more recent iterations including revision to a grand theory,9 this previously developed, middle-range theory was more amenable to the purpose of the review—closely analyzing and categorizing student learning outcomes. The 2005 framework aptly demonstrated the relationship between student, the intervention of simulation, and learner outcomes.8 As the focus of this review was on student learning outcomes achieved through virtual simulation, the following 5 outcomes depicted in the simulation model were placed in a data abstraction form as a foundation to analyze the outcomes of the existing research: learning (knowledge), skill performance, learner satisfaction, critical thinking, and self-confidence. Learning outcomes from each of the 80 studies were categorized into one or several of the 5 outcomes delineated in this framework.8
Data Synthesis
To synthesize the data, the lead researcher read and re-read through the summary tables noting the different learning objectives and contexts, study characteristics, and learning outcomes. Data from the CASP4 tools were aggregated to identify consistent areas of bias identified in the body of evidence. Studies were ranked based on Melnyk and Fineout-Overholt's levels of evidence7 and categorized into high- or lower-quality research. Data regarding learning outcomes were obtained from the extraction table and aggregated in terms of frequency as well as effect on learning.
RESULTS
Study Characteristics
The research studies (N = 80) reviewed spanned 22 years (Fig. 2) and represented the following 15 countries: Australia (n = 2), Canada (n = 6), China (n = 1), Finland (n = 1), Iran (n = 1), Malta (n = 1), Norway (n = 2), Portugal (n = 1), Singapore (n = 2), Spain (n = 1), Sweden (n = 3), Thailand (n = 2), Turkey (n = 2), UK (n = 6), and the United States (n = 49). The studies included nursing students at the preregistration/undergraduate/prelicensure/associate degree/diploma level and graduate levels (Fig. 3). All studies involved samples of nursing students to meet inclusion criteria, and several studies (n = 7) also included participation of students from outside disciplines such as medical, social work, occupational therapy, physician assistant, physical therapy, and pharmacy students. Most studies were conducted at a single site (n = 70), several studies involved 2 sites (n = 5), and several studies involved 3 or more sites (n = 5).
FIGURE 2:
Research study publications by year.
FIGURE 3:
Level of learner of the reviewed studies.
The virtual simulation technology used in these studies also highly varied. Virtual simulations included Second Life, Mooshak, Voki classroom, Digital Clinical Experience, The Neighborhood, virtual ward, computer-assisted learning module, virtual game, vSim for Nursing, Pulse using Adobe Flash, FIRST2 ACTWeb, Web-based, screen-based computer simulation, Microsoft Kinect, CliniSpace, Web-SP Virtual Patient system, Virtual Interactive Practice, CD ROM, CathSim, The Virtual Patient, Virtual Pediatric Patients, e-RAPIDS, virtual animations, VI-MED, virtual simulations using the platform of Unity 3D, eWARD, Virtual Gaming Simulation (VGS), and Virtual Simulation Experience (VSE). Some technologies were available for purchase and others were developed by the faculty member/researcher.
Impact of Virtual Simulation on Student Learning Outcomes
Most evidence (n = 69 studies, 86%) suggested that the intervention of virtual simulation resulted in improved student learning outcomes. The amount of use or time spent in virtual simulation correlated with greater learning benefits.10–12 The learning outcomes captured were categorized within the outcomes of the Jeffries' (2005) Simulation Framework (Fig. 4). Each study design, sampling strategy, type of virtual simulation used, sample characteristics, data collection methods, interventions, major results, and level of evidence were extracted and consolidated into a summary table (see Table, Supplemental Digital Content 1, which demonstrates key study features of all 80 articles reviewed, https://links.lww.com/SIH/A471).
FIGURE 4:
Number of times the outcomes were investigated in the reviewed studies.
Learning (knowledge)
Forty-seven (59%) of the 80 studies specifically sought to examine the outcomes of improving student learning or knowledge. This type of learning fell into the cognitive and affective domains. The learning outcomes were highly varied in terms of context and settings. Learning outcomes were described in the following ways in a cognitive context: knowledge,1,13–29 meaningful learning,30 cognitive gains,31,32 discovering,33 improving learning,6,34 and improved academic performance.35 The learning occurred in a variety of contexts including health assessment,18,36 life support and clinical deterioration,15,37 disaster and decontamination training,37–39 leadership,22 communication,40–42 evidence-based practice,14 poverty,43 patient interviewing,44 mental health,45,46 pediatrics,28,47 chronic care,48 public health,49 and interprofessional education.42,50,51
In addition to learning within the cognitive domain, learning was described in the affective domain. Learning that encompassed attitudes, values, or student engagement was placed in this domain. Virtual simulation was found to improve cultural awareness and competency,52–54 attitudes,18,28,42,43,50,55 value for evidence-based practice,14 emotional connection,56 perceived utility,15 and engagement.14,15,34,57–59 Twelve studies examined student perceptions of learning as opposed to demonstrated learning outcomes. Of the randomized controlled trials (RCTs) that examined learning (knowledge), 13 of the 15 studies demonstrated increases in learning (Table 1). The remaining 2 studies did not examine a pre-post effect of virtual simulation on learning so this could not be determined from the data reported. Most evidence indicated that virtual simulation lead to improved learning outcomes.
TABLE 1:
Results of RCTs Applied to Jeffries' (2005) Framework
Skill Performance
Twenty-nine (36%) of the studies examined the outcomes of skills resulting from simulation. Skills included handwashing,13 health assessment,18,36 life support and deterioration training,15–36,61,64 surgery,62 decontamination,38,39 communication performance,22,65 intravenous insertion,25 nasogastric tube insertion,60 catheter insertion,66 medication administration,63 critical nursing tasks or functions,1,67 and clinical and team performance,29–50,61,64 and Objective Structured Clinical Examination performance.1,13,15,29 Five studies examined student perceptions of skill acquisition rather than demonstrated skills performance. Cohorts that were taught via virtual simulation compared with traditional methods demonstrated a decreased time to perform skills.1,38 Of 13 RCT's that examined skill performance, 8 studies showed statistically significant improvements in skills in the virtual simulation cohort compared with the traditional methods cohort. Most studies demonstrated skill improvement resulting from virtual simulation.
Learner Satisfaction
Forty-one studies (51%) examined learner satisfaction after use of virtual simulation. Studies examined the student experience, appreciation, and perceived benefits of virtual simulation. Students described the virtual simulation experience as enjoyable,68–70 fun,19,39 easy to use,71,72 great,73 realistic,74,75 engaging,10,19,35,39,68,74,76 and express a high level of satisfaction.6,15,18,20,24,25,37,41,58–60,63,76–80 Students enjoyed the ability to interact with others and work in groups.70 Students appreciated the ability to study at home and felt that the virtual simulations helped more than just reading from textbooks.28 After virtual simulation, they felt better prepared.44,76,81 Gerdprasert et al28 found that the more the time students spent on the Web site, the higher the student satisfaction scores. Of 12 RCT's that examined satisfaction, 10 studies incorporating virtual simulation reported high student satisfaction.
Critical Thinking
Ten (12.5%) studies examined the outcome of critical thinking. As this term is interpreted differently and has many forms of measurement, a broad interpretation was applied. For example, terms reflecting critical thinking included clinical decision-making,1,12,24,34,47,57,62 clinical judgment,24 critical nursing tasks,1 recognizing clinical deterioration,15 recognizing poverty,43 and clinical problem-solving.75 Of the 5 RCT's that examined critical thinking, 3 demonstrated a result of no difference in critical thinking compared with traditional methods,43,47,62 whereas 2 studies suggested a statistically significant increase in the virtual simulation cohort.1,15
Self-confidence
Thirteen (16%) studies examined the outcome of self-confidence resulting from virtual simulation.24–26,29,32,33,36,38,40,47,54,59,72 The terms of self-confidence and self-efficacy both were included in this interpretation. Most studies demonstrated an increase of self-confidence resulting from virtual simulation.24,26,29,33,54,59,72 When comparing virtual simulation to traditional methods, several studies found that no significant differences in self-confidence were obtained.25,36,38,47 Of the RCT's examining self-confidence, the results were mixed with 2 of 6 studies demonstrating an increase in self-confidence,38,64 2 studies reporting no difference compared with the traditional methods cohort,37,47 and 1 study demonstrating lower self-confidence scores in comparison with the traditional methods group.32
Synthesis of the Results, Biases, and Gaps
Summary of Evidence
When applying the Levels of Evidence defined by Melnyk and Fineout-Overholt (2015),7 18 studies were rated level 2, 22 studies were rated level 3, 5 studies were rated level 4, and 35 studies were rated as level 6. Therefore, half (n = 40, 50%) of the evidence was of high quality and the other half (n = 40, 50%) was of lower quality. When honing in on only the level 2 studies or the highest levels of evidence that examined the intervention of virtual simulation (n = 17), RCTs, most studies (n = 13, 76%) demonstrated that virtual simulation lead to statistically significant gains in outcomes when compared with traditional methods. One RCT was not included in the synthesis because it compared debriefing methods as opposed to the intervention of virtual simulation versus traditional methods.26 One study, conducted in 2003, indicated that traditional methods lead to better learning outcomes when compared with virtual simulation.32
Technical Issues
As virtual simulation involves technology, technical issues were noted as a common problem. Ten studies reported that technical issues were an interference to the learning experience.22,39,46,47,49–51,71,82,83 The technical issues caused anxiety,22 frustration,40,71 and dissatisfaction47 for students.
Risk of Bias Across Studies
To assess risk of bias, specific CASP tools were used based on the type of study conducted. For cohort studies, the areas of most potential for bias were reflected in (a) lack of large samples, (b) lack of the identification of confounding factors, and (c) lack of reporting confidence intervals. For randomized controlled studies, there was risk for bias most often with respect to (a) failing to identify the effect size as well as (b) not blinding study personnel to the intervention. In the qualitative studies reviewed, the most common areas with potential for bias were that (a) the relationship between researcher (who was often the teacher) and participants was not adequately considered and (b) the data analyses lacked sufficient rigor.
Conflicting Findings
Although evidence demonstrated virtual simulation as superior to traditional methods in relation to improving learning (knowledge) outcomes, 8 of the 15 RCTs examining learning (knowledge) found no statistically significant differences in outcomes between the control group and intervention group. Although virtual simulation was effective in improving learning outcomes, the effectiveness, at times, was equivalent to the traditional methods.
There were also conflicting findings regarding the retention of learning over time. Two studies found that the virtual simulation group demonstrated improved retention of learning over time in comparison with the control group.13–37,61,64 Two different studies found no differences in retention of learning over time when virtual simulation was compared with traditional methods.15,31 Therefore, it remains unclear whether virtual simulation leads to improved retention over time. As the amount of time and doses of virtual simulation highly varied throughout the literature, there was limited information about the amount of time of virtual simulation needed to achieve desired learning outcomes.
Gaps
As virtual simulation is a relatively new science, a number of gaps were noted with respect to the current body of evidence. Only one study was located that examined the cost of virtual simulation compared with manikin-based simulation.24 Haerling24 (2018) calculated that manikin-based simulation costs were US $36.55 per student, whereas virtual simulation costs were US $10.89 per student yielding a cost/utility ratio of US $3.62 for manikin-based simulation and US $1.08 for virtual simulation. The literature was scant with respect to cost-benefit analyses in the realm of virtual simulation.
As debriefing is a major part of simulation and learning, the literature in virtual simulation demonstrated high variability with respect to debriefing practices. Most studies did not mention debriefing practices, although, with some virtual simulation products, students are debriefed by the game. Verkuyl et al26 (2018) conducted a randomized controlled study comparing 3 debriefing methods after a virtual simulation. They found that all groups made significant knowledge and self-efficacy gains with no statistically significant differences in outcomes among in person, virtual, and self-debriefing. There is little evidence regarding debriefing virtual simulation.
After examining what was known about virtual simulation, there was a noted lack of knowledge regarding best practices in the methods of administering virtual simulation as well as the adequate exposure time (dose) to result in significant learning outcomes. Second, there was very little information reported about differences in learning depending on the age or generation of the learner, despite known differences in learning styles and generational learning preferences. Third, it is unknown whether using virtual simulation in conjunction with traditional simulation is advantageous. Fourth, there is little evidence evaluating use of virtual simulation at a program level. Fifth, it is unknown whether virtual simulation could be substituted for clinical practicum with comparable outcomes. Finally, there is a lack of studies investigating whether learning achieved through virtual simulation translates to clinical practice or ultimately impacts patient outcomes.
DISCUSSION
The results of this review suggest that most evidence (86%) indicated that virtual simulation positively impacted student learning outcomes. This review demonstrated that virtual simulation has been used effectively to improve learning (knowledge), skills/performance, critical thinking, self-confidence, and provide learner satisfaction. The remaining 14% of studies did not necessarily indicate that virtual simulation decreased learning outcomes; rather, some lower level studies such as descriptive studies or feasibility studies did not examine an outcome variable of learning; thus, they could not be included. In addition, some studies found no difference between virtual simulation and a comparative traditional method; however, ultimately, both modalities resulted in improved learning. Only one study demonstrated that traditional methods performed better than virtual simulation.32 Overall, the supportive body of evidence unveiled may be helpful to guide faculty, administrators, and policymakers in decisions regarding virtual simulation.
Summary of Evidence
The learning that occurred as well as the offered contexts and modalities was very broad and highly varied making it challenging to mathematically synthesize or determine best practices or standards in virtual simulation at this time. More consistent methods in terms of learning objectives, virtual simulation modality, and virtual simulation exposure time in relation to select desired learning outcomes are necessary to forward the science. Effect sizes and confidence intervals were rarely reported. Given the current state of the science, multisite, longitudinal studies using randomized controlled designs are needed. We also suggest comparative studies to examine the effectiveness of different forms of education on learning outcomes.
In comparison with manikin-based simulation, the learning outcomes were similar. Most of the learning outcomes were in the cognitive domain, although several studies demonstrated that virtual simulation corresponded with an improved psychomotor skill performance afterward. Because of the computer-based nature of virtual simulation, it is not surprising to see that many studies examined cognitive outcomes including communication and test performance. Interestingly, nearly 20 studies examined learning in the affective domain. Learner attitudes, perceived utility and engagement, and value for the virtual simulation were affected in various studies. This is an important benefit as affecting students' feelings and values is likely to affect future actions—a domain that could be easily missed in nursing education. Critical thinking was the least measured of the Jeffries' 5 outcomes—perhaps, because of the difficulty of measuring this complex phenomenon. On the other hand, several studies used Objective Structured Clinical Examination1,13,16,29 performance as a learning outcome, which arguably includes a subcomponent of critical thinking.
In the evaluation of learning outcomes, the term “engagement” was placed under the category of learning/knowledge. Although many would argue that engagement and learning are completely different concepts, engagement was placed under learning/knowledge because learning occurs best when one is engaged and engagement reflects the affective domain of recognizing the importance or value of the simulation—another facilitator of learning. Engagement may have been better placed as a separate outcome of learning outside of the 5 delineated outcomes in Jeffries' (2005) framework as this framework is not intended to be exhaustive of all learning outcomes.
A large number of studies examined perceptions of learning and learner satisfaction rather than demonstrated learning outcomes. As the science of virtual simulation is relatively new compared with manikin-based simulation, this finding is not surprising as these types of descriptive studies are often starting points. Moving forward, simulation researchers should consider evaluating more objective evidence using quantitative methods.
Various studies described challenges with the technology.22,39,46,47,49–51,71,82,83 Log-in problems, inability to multitask, lag time, lack of realism, and other issues caused student anxiety and frustration. Therefore, when conducting virtual simulation research, providing ample technological support, orientation, time to “play,” and troubleshoot is important on the front-end to avoid a disappointing and ineffective learning experience.
Based on the analysis of bias across studies, virtual simulation researchers should consider the following when designing future studies:
- Anticipate challenges with technology
- Identify the effect size/s
- Conduct a power analysis to determine adequate sample size
- Blind study personnel
- Identify confounding factors
- Report confidence intervals
- Examine hard metrics related to student learning as opposed to perceptions of learning
To conduct more robust simulation research in the field, applying guidelines set by Issenberg et al84 and Cheng et al85 are recommended.
Definition of Virtual Simulation
It is apparent that there lies discrepancy in the meaning of the term virtual simulation. Kardong-Edgren et al (2019) describe the confusion resulting from the various definitions and call for more precise definitions.86 The breadth of this term may lead to various interpretations. We noted that the terms virtual simulation and virtual reality were used interchangeably in the earlier literature but have since evolved to signify different concepts. As the nomenclature of virtual simulation is unclear, Cant et al (2019) recommend that terminologies be refined to include expressive description of the simulation components including (a) level of fidelity, (b) immersion, and (c) bodily form of the patient.2
After this extensive review of the literature, we propose that the simulation community better differentiate the terms of virtual simulation, virtual reality, mixed reality, and augmented reality. These terms all have distinctly different meanings. We suggest that virtual simulation should no longer be used as an umbrella term to describe any and every virtual modality; rather, it should be used to refer to partially immersive, screen-based experiences as this is the interpretation most reflected in the current literature. On a different note, virtual reality or virtual reality simulation has evolved to represent technology that offers a fully immersive experience through the use of a headset that covers the eyes, whereas the term augmented reality has evolved to mean the juxtaposition or overlay of a digital learning environment with real life. Although early literature demonstrates interchangeable use of these terms, the science of simulation has since advanced. Better distinction and clarity of these terms in the Healthcare Simulation Dictionary are warranted to deepen our current understanding.
Future Research
Given the gaps identified from this review, the following areas for research are encouraged as priorities: (a) examine the effects of virtual simulation when integrated throughout the curriculum; (b) examine the effects of virtual simulation when used in conjunction with manikin-based simulation; (c) determine adequate amounts of time and dosing of virtual simulation to result in desired learning outcomes; (d) examine retention of learning resulting from virtual simulation longitudinally; and (e) examine whether virtual simulation may be substituted for a select amount of clinical practicum. Furthermore, although stakeholders in simulation and nursing education may benefit from the results of a meta-analysis, the high number of exploratory, descriptive, feasibility, and usability studies as well as the high variability of study objectives, conditions, equipment, and samples present significant barriers. We urge simulation researchers to conduct RCTs when possible to elevate the science of virtual simulation.
Limitations
This review was limited in several ways. Adding search terms such as “serious games,” “videogames,” and “screen-based learning” could have expanded the findings. However, these terms were not used as the goal was to focus on society's interpretation of virtual simulation. Moreover, no gray literature searches were conducted. Thus, because only peer-reviewed research studies were included, findings may overreport positively significant findings. Because of the vast breadth of this review, the various study designs combined, and various contexts in the use of virtual simulation, a meta-analysis could not be conducted. The focus on nursing education as opposed to health professions education may be determined a limitation as well as a strength.
CONCLUSIONS
Virtual simulation is a relatively new pedagogy in the context of nursing education that has been demonstrated to improve student learning outcomes. The body of evidence indicates that virtual simulation improves learning outcomes. As the relatively new science of virtual simulation progresses, more evidence is needed to substantiate best practices in methodology of virtual simulation-based education. Comparative studies involving virtual simulation versus other learning modalities are indicated to inform evidence-based teaching. Virtual simulation researchers may consider the analysis of bias across studies from this review and work to better decrease risk for bias in future work through improving study design and applying simulation research guidelines.85 As future generations of students emerge as digital natives and technology continues to progress, virtual simulation is a promising pedagogy of the now and the future.
ACKNOWLEDGMENTS
The activities reported here were supported (in part) by the Josiah Macy Jr. Foundation.
REFERENCES
1. Leflore JL, Anderson M, Zielke MA, et al. Can a virtual patient trainer teach student nurses how to save lives—teaching nursing students about pediatric respiratory diseases.
Simul Healthc 2012;7(1):10–17.
2. Cant R, Cooper S, Sussex R, Bogossian F. What's in a name? Clarifying the nomenclature of virtual simulation.
Clin Simul Nurs 2019;27:26–30.
3. Lopreiato JO.
Heathcare Simulation Dictionary. Rockville, MD: Agency for Healthcare Research and Quality; 2016:1–48.
4. Critical Appraisal Skills Programme. CASP checklist. Updated 2017. Available at:
https://casp-uk.net/. Accessed December 11, 2017.
5. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
PLoS Med 2009; 6(7):e1000097.
6. Foronda CL, Swoboda SM, Henry MN, Kamau E, Sullivan N, Hudson KW. Student preferences and perceptions of learning from vSim for nursing.
Nurs Educ Pract 2018;33:27–32.
7. Melnyk BM, Fineout-Overholt E.
Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice. 3rd ed. Philadelphia: Wolters Kluwer Health; 2015.
8. Jeffries PR. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. 2005;26(2):96–103.
9. Jeffries PR.
The NLN/Jeffries Simulation Theory. Philadelphia: Wolters Kluwer; 2015.
10. Carlson Sabelli, Linnea L, Giddens JF, Fogg L, Fiedler RA. Challenges and benefits of using a virtual community to explore nursing concepts among baccalaureate nursing students.
Int J Nurse Educ Scholarsh 2011;8(1):1–14.
11. Giddens J, Fogg L, Carlson-Sabelli L. Learning and engagement with a virtual community by undergraduate nursing students.
Nurs Outlook 2010;58(5):261–267.
12. Kleinheksel AJ. Transformative learning through virtual patient simulations: predicting critical student reflections.
Clin Simul Nurs 2014;10(6):e301–e308.
13. Bloomfield J, Roberts J, While A. The effect of computer-assisted learning versus conventional teaching methods on the acquisition and retention of handwashing theory and skills in pre-qualification nursing students: a randomised controlled trial.
Int J Nurs Stud 2010;47(3):287–294.
14. Foronda CL, Hudson KW, Budhathoki C. Use of virtual simulation to impact nursing students' cognitive and affective knowledge of evidence-based practice.
Worldviews Evid Based Nurs 2017;14(2):168–170.
15. Liaw SY, Chan SW, Chen F, Hooi SC, Siau C. Comparison of virtual patient simulation with mannequin-based simulation for improving clinical performances in assessing and managing clinical deterioration: randomized controlled trial.
J Med Internet Res 2014;16(9):e214.
16. Mosalanejad L, Shahsavari S, Sobhanian S, Dastpak M. The effect of virtual versus traditional learning in achieving competency-based skills.
Turk Online J Dist Educ 2012;13(2):69–75.
17. Gu Y, Zou Z, Chen X. The effects of vSIM for nursing™ as a teaching strategy on fundamentals of nursing education in undergraduates.
Clin Simul Nurs 2017;13(4):194–197.
18. Kaveevivitchai C, Chuengkriankrai B, Luecha Y, Thanooruk R, Panijpan B, Ruenwongsa P. Enhancing nursing students' skills in vital signs assessment by using multimedia computer-assisted learning with integrated content of anatomy and physiology.
Nurse Educ Today 2009;29(1):65–72.
19. Chia P. Using a virtual game to enhance simulation based learning in nursing education.
Singapore Nurs 2013;40(3):21–26.
20. Tschannen D, Dorn R, Tedesco C. Improving knowledge and behavior of leadership and followership among the interprofessional team.
Int J Med Educ 2018;9:182–188.
21. Broom M. Using online simulation in child health nurse education: Mark broom and colleagues report on using a computerised 'virtual' ward with nursing students before clinical placements (report).
Paediatr Nurs 2009;21(8):32.
22. Foronda C, Budhathoki C, Salani D. Use of multiuser, high-fidelity virtual simulation to teach leadership styles to nursing students.
Nurse Educ 2014;39(5):209–211.
23. Borg Sapiano A, Sammut R, Trapani J. The effectiveness of virtual simulation in improving student nurses' knowledge and performance during patient deterioration: a pre and post test design.
Nurse Educ Today 2018;62:128–133.
24. Haerling KA. Cost-utility analysis of virtual and mannequin-based simulation.
Simul Healthc 2018;13(1):33–40.
25. Ismailoglu EG, Zaybak A. Comparison of the effectiveness of a virtual simulator with a plastic arm model in teaching intravenous catheter insertion skills.
Comput Inform Nurs 2018;36(2):98–105.
26. Verkuyl M, Atack L, McCulloch T, et al. Comparison of debriefing methods after a virtual simulation: an experiment.
Clin Sim in Nurs 2018;19:1–7.
27. Tschannen D, Aebersold M, McLaughlin E, Bowen J, Fairchild J. Use of virtual simulations for improving knowledge transfer among baccalaureate nursing students.
J Nurs Educ Pract 2012;2(3):15–24.
28. Gerdprasert S, Pruksacheva T, Panijpan B, Ruenwongsa P. An interactive web-based learning unit to facilitate and improve intrapartum nursing care of nursing students.
Nurse Educ Today 2011;31(5):531–535.
29. Cooper S, Cant R, Bogossian F, Kinsman L, Bucknall T. Patient deterioration education: evaluation of face-to-face simulation and e-simulation approaches.
Clin Simul Nurs 2015;11(2):97–105.
30. Poikela P, Ruokamo H, Teräs M. Comparison of meaningful learning characteristics in simulated nursing practice after traditional versus computer-based simulation method: a qualitative videography study.
Nurse Educ Today 2015;35(2):373–382.
31. Fernández Alemán JL, Carrillo de Gea JM, Rodríguez Mondéjar JJ. Effects of competitive computer-assisted learning versus conventional teaching methods on the acquisition and retention of knowledge in medical surgical nursing students.
Nurse Educ Today 2011;31(8):866–871.
32. Engum SA, Jeffries P, Fisher L. Intravenous catheter training system: computer-based education versus traditional learning methods.
Am J Surg 2003;186:67–74.
33. Verkuyl M, Lapum JL, Hughes M, et al. Virtual gaming simulation: exploring self-debriefing, virtual debriefing, and in-person debriefing.
Clin Simul Nurs 2018;20:7–14.
34. Mccallum J, Ness V, Price T. Exploring nursing students' decision-making skills whilst in a second life clinical simulation laboratory.
Nurse Educ Today 2011;31(7):699–704.
35. Giddens J, Hrabe D, Carlson-Sabelli L, Fogg L, North S. The impact of a virtual community on student engagement and academic performance among baccalaureate nursing students.
J Prof Nurs 2012;28(5):284–290.
36. Bryant R, Miller CL, Henderson D. Virtual clinical simulations in an online advanced health appraisal course.
Clin Simul Nurs 2015;11(10):437–444.
37. Farra S, Miller E, Timm N, Schafer J. Improved training for disasters using 3-D virtual reality simulation.
West J Nurs Res 2013;35(5):655–671.
38. Farra SL, Smith S, Gillespie GL, et al. Decontamination training.
Adv Emerg Nurs J 2015;37(2):125–133.
39. Ulrich D, Farra S, Smith S, Hodgson E. The student experience using virtual reality simulation to teach decontamination.
Clin Simul Nurs 2014;10(11):546–553.
40. Foronda C, Lippincott C, Gattamorta K. Evaluation of virtual simulation in a master's-level nurse education certificate program.
Comput Inform Nurs 2014;32(11):516–522.
41. Gobbi M, Monger E, Watkinson G, et al. Virtual interactive practice: a strategy to enhance learning and competence in health care students.
Stud Health Technol Inform 2004;107:874.
42. Sweigart LI, Umoren RA, Scott PJ, et al. Virtual TeamSTEPPS(®) simulations produce teamwork attitude changes among health professions students.
J Nurs Educ 2016;55(1):31–35.
43. Menzel N, Willson LH, Doolen J. Effectiveness of a poverty simulation in second life: changing nursing student attitudes toward poor people.
Int J Nurse Educ Scholarsh 2014;11(1):1–7.
44. Sweigart L, Hodson-Carlton R, Campbell BR, Lutz DR. Second life environment: a venue for interview skill development.
CIN 2010;28(5):258–263.
45. Sunnqvist C, Karlsson K, Lindell L, Fors U. Virtual patient simulation in psychiatric care—a pilot study of digital support for collaborate learning.
Nurse Educ Pract 2016;17:30–35.
46. Kidd L, Knisley S, Morgan K. Effectiveness of a second life® simulation for undergraduate mental health nursing students.
Clin Simul Nurs 2012;8(8):e408.
47. Cobbett S, Snelgrove-Clarke E. Virtual versus face-to-face clinical simulation in relation to student knowledge, anxiety, and self-confidence in maternal-newborn nursing: A randomized controlled trial.
Nurse Educ Today 2016;45:179–184.
48. Tilton KJ, Tiffany J, Hoglund BA. Non-acute-care virtual simulation: preparing students to provide chronic illness care.
Nurs Educ Perspect 2015;36(6):394–395.
49. Schaffer MA, Tiffany JM, Kantack K, Anderson LJW. Second life(®) virtual learning in public health nursing.
J Nurs Educ 2016;55(9):536–540.
50. Caylor S, Aebersold M, Lapham J, Carlson E. The use of virtual simulation and a modified TeamSTEPPS™ training for multiprofessional education.
Clin Simul Nurs 2015;11(3):163–171.
51. Seefeldt TM, Mort JR, Brockevelt B, et al. A pilot study of interprofessional case discussions for health professions students using the virtual world second life.
Curr Pharm Teach Learn 2012;4(4):224–231.
52. Tiffany JM, Hoglund BA. Using virtual simulation to teach inclusivity: a case study.
Clin Simul Nurs 2016;12(4):115–122.
53. Giddens JF, North S, Carlson-Sabelli L, Rogers E, Fogg L. Using a virtual community to enhance cultural awareness.
J Transcult Nurs 2012;23(2):198–204.
54. Weideman YL, Young L, Lockhart JS, Grund FJ, Fridline MM, Panas M. Strengthening cultural competence in prenatal care with a virtual community: building capacity through collaboration.
J Prof Nurs 2016;32(5):S53.
55. Strang Zook S, Hulton LJ, Dudding CC, Stewart AL, Graham AC. Scaffolding interprofessional education: unfolding case studies, virtual world simulations, and patient-centered care.
Nurse Educ 2018;43(2):87–91.
56. Shuster G, Giddens JF, Roehrig N. Emotional connection and integration: dominant themes among undergraduate nursing students using a virtual community.
J Nurs Educ 2011;50(4):222–225.
57. Burke SM. Cultivating critical thinking using virtual interactive case studies.
J Pediatr Nurs 2017;33:94–96.
58. Fowler T, Phillips S, Patel S, et al. Virtual interprofessional learning.
J Nurs Educ 2018;57(11):668–674.
59. Verkuyl M, Romaniuk D, Mastrilli P. Virtual gaming simulation of a mental health assessment: a usability study.
Nurse Educ Pract 2018;31:83–87.
60. Aebersold M, Voepel-Lewis T, Cherara L, et al. Interactive anatomy-augmented virtual simulation training.
Clin Simul Nurs 2018;15:34–41.
61. Cook NF, Mcaloon T, O'Neill P, Beggs R. Impact of a web based interactive simulation game (PULSE) on nursing students' experience and performance in life support training—a pilot study.
Nurse Educ Today 2012;32(6):714–720.
62. Durmaz A, Dicle A, Cakan E, Cakir Ş. Effect of screen-based computer simulation on knowledge and skill in nursing students' learning of preoperative and postoperative care management: a randomized controlled study.
CIN 2012;30(4):196–203.
63. Jeffries PR. Computer versus lecture: a comparison of two methods of teaching oral medication administration in a nursing skills laboratory.
J Nurs Educ 2001;40(7):323–329.
64. Verkuyl M, Romaniuk D, Atack L, Mastrilli P. Virtual gaming simulation for nursing education: an experiment.
Clin Simul Nurs 2017;13(5):238–244.
65. Strekalova YA, Krieger JL, Kleinheksel AJ, Kotranza A. Empathic communication in virtual education for nursing students.
Nurse Educ 2017;42(1):18–22.
66. Smith PC, Hamilton BK. The effects of virtual reality simulation as a teaching strategy for skills preparation in nursing students.
Clin Simul Nurs 2015;11:52–58.
67. Carman M, Xu S, Rushton S, et al. Use of a virtual learning platform for distance-based simulation in an acute care nurse practitioner curriculum.
Dimens Crit Care Nurs 2017;36(5):284–289.
68. Anderson J, Page A, Wendorf DM. Avatar-assisted case studies.
Nurse Educ 2013;38(3):106–109.
69. Evans DA, Curtis AR. Animosity, antagonism, and avatars: teaching conflict management in second life.
J Nurs Educ 2011;50(11):653–655.
70. Schmidt B, Stewart S. Implementing the virtual world of second life into community nursing theory and clinical courses.
Nurse Educ 2010;35(2):74–78.
71. Foronda CL, Swoboda SM, Hudson KW, et al. Evaluation of vSIM for nursing™: a trial of innovation.
Clin Simul Nurs 2016;12(4):128–131.
72. Tait M, Tait D, Thornton F, Edwards M. Development and evaluation of a critical care e-learning scenario.
Nurse Educ Today 2008;28(8):970–980.
73. Pittiglio LI, Harris MA, Mili F. Development and evaluation of a three-dimensional virtual hospital unit: VI-MED.
CIN 2011;29(5):267–271.
74. Forsberg E, Georg C, Ziegert K, Fors U. Virtual patients for assessment of clinical reasoning in nursing—a pilot study.
Nurse Educ Today 2011;31(8):757–762.
75. Kiegaldie D, White G. "The virtual patient"—development, implementation and evaluation of an innovative computer simulation for postgraduate nursing students.
J Educ Multimedia Hypermedia 2006;15(1):31–47.
76. Verkuyl M, Hughes M, Tsui J, Betts L, St-Amant O, Lapum JL. Virtual gaming simulation in nursing education: a focus group study.
J Nurs Educ 2017;56(5):274.
77. Johannesson E, Olsson M, Petersson G, Silen C. Learning features in computer simulation skills training. (report).
Nurse Educ Pract 2010;10(5):268.
78. Merritt LS, Brauch AN, Bender AK, Kochuck D. Using a web-based e-visit simulation to educate nurse practitioner students. 2018;57(5):304–307.
79. Padilha JM, Machado PP, Ribeiro AL, Ramos JL. Clinical virtual simulation in nursing education.
Clin Simul Nurs 2018;15:13–18.
80. Tjoflat I, Brandeggen TK, Strandberg ES, Dyrstad DN, Husebo SE. Norwegian nursing students' evaluation of vSim for nursing.
Adv Simul 2018;3(10):1–6.
81. Sweigart L, Hodson-Carlton K. Improving student interview skills: the virtual avatar as client.
Nurse Educ 2013;38(1):11–15.
82. Aebersold M, Tschannen D, Stephens M, Anderson P, Lei X. Second life®: a new strategy in educating nursing students.
Clin Simul Nurs 2012;8(9):e475.
83. Tiffany J, Hoglund BA. Teaching/learning in second life: perspectives of future nurse-educators.
Clin Simul Nurs 2014;10(1):e24.
84. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.
Med Teach 2005;27(1):10–28.
85. Cheng A, Kessler D, Mackinnon R, et al. Reporting guidelines for health care simulation research: extensions to the CONSORT and strobe statements.
Simul Healthc 2016;11(4):238–248.
86. Kardong-Edgren SS, Farra SL, Alinier G, Young HM. A call to unify definitions of virtual reality.
Clin Simul Nurs 2019;31:28–34.
