Educational Larp in the Middle School Classroom: A Mixed Method Case Study

Popular abstract: This mixed method case study examines the effectiveness of an educational role-playing (edu-larp) intervention into the science curriculum of a charter school in Los Angeles that serves an economically disadvantaged population. Utilizing psychometric surveys and semi-structured interviews, the investigators gathered data from middle school students evaluating their development along five dimensions of learning before and after the semester-long program: intrinsic motivation, perceived competence, school engagement, team work, and leadership. The study also gathered qualitative data from the 23 students in this convenient sample group regarding their overall experiences with the edu-larp method. When paired with traditional pedagogy, out of these five dimensions of student development, the investigators found that the edu-larp intervention helped increase overall intrinsic motivation and interest/enjoyment of science in the quantitative data. The qualitative and quantitative findings merged to reveal improvement in perceived competence in science. In interviews, students demonstrated a strong belief that larp aided in the development of all five dimensions and expressed universal interest in learning through edu-larp in the future. Overall, the data suggested that adding the edu-larp component to the existing science curriculum impacted the experience of students by increasing interest, engagement


INTRODUCTION
Interest in live action role-playing as an educational tool (edularp) has increased in recent years.Several professional organizations have emerged specializing in edu-larp, including consulting companies, after-school programs, and entire schools devoted to the method.This interest corresponds with other trends in pedagogy that favor an experiential mode of instruction, including outgrowths from drama (Mochocki, 2013), digital gaming (Whitton, 2014), and simulation (Standiford, 2014).Edularp in particular has emerged from the leisure activity of live action role-playing (larp), sharing many of the same benefits afforded from these cousin forms, while still remaining somewhat unique due to its roots in popular culture.
Larp is a form of game play in which participants physically embody characters within a fictional scenario for extended periods of time.Designers can set larps in any time, place, or genre.Characters range from strongly similar to the player's primary identity to completely distinct (Bowman, 2010).As a pedagogical outgrowth, edu-larp refers to an educational roleplaying exercise in which participants adopt a new role for a long period of time in a bounded, fictional scenario that may or may not resemble mundane reality.Some edu-larp scenarios contain rules or win conditions, but not all.
The efficacy of experiential modes of pedagogy remain difficult to study.While teachers and students alike often express enthusiasm for various forms of hands-on, experiential learning, measuring the effects of these interventions in a statistically satisfying way can prove challenging.First, the immersive experience of a method like edu-larp trains multiple skills at once rather than imparting facts in a direct delivery fashion (Henriksen, 2006;Porter, 2008;Blatner, 2009;Hyltoft, 2010).In many edu-larps, students are embodying a role other than their primary identity in a new, fictional scenario.This enactment involves processing multiple levels of social information -including status hierarchies and socio-historical factors -in addition to subject matter knowledge, which educators can embed in the larp from any discipline (Hyltoft, 2008(Hyltoft, , 2012)).Second, well-designed edu-larps allow for a level of agency and creativity that empowers the students to make decisions in the fictional universe.Contact with subject matter facts occurs alongside this flow of creative embodiment, which can lead to somewhat amorphous or unpredictable responses in exit interviews; in other words, students may perceive the exercise as "fun" without fully reflecting upon their own learning (Henriksen, 2006;Harder, 2007;Henriksen, 2008).Third, if a student improved in their comprehension of the science curriculum, for example, how do educators determine whether that comprehension was gained during the traditional classroom education, as the result of the edu-larp education, or due to unrelated changes in their individual cognitive development?
As a consequence of these complications, current edu-larp researchers emphasize the need for embedding reliable assessment techniques into the exercises or after them (Cherif and Somervill, 1995 Mochocki 2013a, 101-111).These assessments range from self-reflections to tests.While educators have not reached a consensus regarding the best method of assessment, most emphasize the need for thorough briefing sessions before and debriefing after the experience to help cement the lessons the teachers intended for students to internalize (Cherif and Somervill, 1995 Andresen ed., 2012, p. 17; Harviainen, Lainema, and Saarinen, 2012, p. 10-11;Hyltoft, 2012, p. 21;Karalevich, 2012, p. 42;Harviainen and Savonsaari, 2013).
While research methods to establish a direct correlation between edu-larp and the retention of specific subject matter require further design, several dimensions of "soft skill" development are easier to measure.Experiential learning is often praised for increasing intrinsic motivation, as the students are granted agency within the exercise to perform actions rather than passively receiving material, which encourages a desire for success (Whitton, 2014;Hyltoft, 2010Hyltoft, , 2012)).Researchers also claim that edu-larp specifically increases self-efficacy and perceived competence (Blatner, 2009;Balzac, 2011;Hoge, 2013, p. 48-49).Additionally, the method may increase school engagement in students, as facts take on new meanings within the fictional setting and the role encourages them to see connections between the subject matter and potential practical application (Howes and Cruz, 2009, p. 34).Edu-larp and other role-playing exercises are also often used as ways to increase team work in various settings (Sancho, et al., 2009, p. 111;Hoge, p. 2013, 48), as well as platforms to practice leadership skills (Hyltoft p. 2010, p. 53).
As researchers cite edu-larp as improving many dimensions of student development (see Figure 1), this mixed method exploratory case study examines a class of middle school students according to five factors: intrinsic motivation, perceived competence, school engagement, team work, and leadership.The study explores the influence of a semester-long, science edu-larp curriculum on middle school students in a disadvantaged area of Los Angeles by collecting data from observational reports, quantitative surveys, and qualitative interviews.
The investigators also gathered data on media engagement in order to ascertain potential similarities in interests with larp genres or barriers to entry into the content of the intervention.The research questions are as follows: 1. Do self-reported scores in intrinsic motivation, perceived competence, school engagement, team work, and leadership change in the participants after the addition of edu-larp to the curriculum? 2. How does adding an edu-larp component into a science curriculum impact the scholastic and social experience of the participants?

In what ways do the quantitative and qualitative findings converge?
The goal of this study was to gather thorough data from students assessing their own development along these various dimensions before and after the intervention, as well as to evaluate their overall experiences with the edu-larp method.The investigators hypothesized that the edu-larp intervention would help increase these five dimensions of student development when paired with traditional pedagogy, although the design was exploratory in nature.
Other case studies have attempted to measure the pedagogical benefits of experiential learning, as shown in the literature review below.However, to the investigators' knowledge, no study exists that has collected mixed method data on an edu-larp intervention according to all five of these dimensions.
Out of these five hypothesized dimensions, the quantitative data revealed that students' overall intrinsic motivation and interest/enjoyment of science significantly increased over the course of the semester.The qualitative and quantitative findings converged to reveal an increase in perceived competence in science.In the ethnographic interviews, students expressed a strong belief that larp aided in the development of all five dimensions and a unanimous interest in learning through edularp in the future.The data did not reveal significant changes in the areas of team work and leadership between the two time points, although several students offered examples in which these two factors were exercised during the course of the edularps.
The quantitative results remain limited due to the lack of availability of a control group and the small sample size.In addition, the quantitative questions did not evaluate the effectiveness of edu-larp specifically, but rather asked students to measure their overall competencies in these areas over time.Therefore, conclusions cannot be drawn about edu-larp as a specific factor from the quantitative data alone, as students also experienced traditional pedagogy curriculum during the time of the study.However, the qualitative interviews, which asked specific questions regarding the edu-larp pedagogy, revealed a strong student belief in the effectiveness of the method along all the dimensions of learning and a great degree of enthusiasm for the method.Overall, the conclusions of this exploratory pilot study are not generalizable, but may serve as a useful starting point for future edu-larp research.

LITERATURE REVIEW
Literature extolling the benefits of role-playing as an experimental training method are already prevalent in several disciplines unrelated to the larp leisure activity.Edu-larp follows the theoretical principles of the educational theories of experiential learning (Kolb, 1984) and situated learning (Lave and Wenger, 1991).According to Kolb (1984), experiential learning posits that knowledge is acquired through concrete experience (doing), reflective observation (thinking back to the experience), abstract conceptualization (forming a theory about what was observed), and active experimentation (testing the new theory).In situated learning, Lave and Wenger (1991) explain that learning does not occur in a vacuum and is socially coconstructed in a dynamic physical environment.Furthermore, Dreyfus and Dreyfus (1980) theorize through the use of hermeneutic phenomenology that expertise is gained through extensive experience rather than solely from book knowledge.Bowman (2014) provides an extensive, if not exhaustive, review of the benefits of experiential learning streamlined with current research in edu-larp.This literature review includes segments of that larger chapter, providing a condensed summary of these various dimensions of student learning (see Figure 1) and a consideration of the significance of edu-larp within a larger context, similar to previous work by Kot (2012).
Several forms of experiential learning currently exist in pedagogy.Game-based learning is a f o rm o f e d u c a t io n t h a t includes systems for success and failure, but may not involve a role.Simulations attempt to replicate real world scenarios in low-consequence contexts and generally involve some degree of role.Drama is a form of theatrical enactment that avoids extensive scripting in favor of role-based, collaborative improvisation into a fictional situation.Role-playing refers to the act of adopting a new role for a long period of time in a bounded, fictional scenario that may or may not resemble mundane reality.Role-playing games include systems for bounding the reality of this fiction, often including rules for success and failure.Edularp, therefore, has all the properties of a role-playing scenario and may or may not feature systems for success and failure; the edu-larp literature includes scenarios that are purely experiential and ones with game systems.
Ultimately, each of these categories overlap to some degree.Therefore, for the purpose of this review, literature regarding educational drama, simulation, game-based learning, and edularp are considered alongside one another as cousins (Mochocki, 2013), indicating similarities within the discourse regarding their benefits.Also, this review only covers English-speaking literature, while several useful sources exist in other languages.Though many of these articles on experiential learning feature theoretical concepts or anecdotal data, a few studies have conducted more rigorous assessments, as noted at the end of this section.In the leisure setting, role-playing games offer the benefits of creating community, improving problem solving, enhancing social skills, and providing opportunities for identity exploration (Bowman, 2010).Some larps focus on fantasy, science fiction, horror, or post-apocalyptic genres, whereas the Nordic larp tradition often tackles real world social issues through intense, immersive scenarios, attempting to raise consciousness on important topics from multiple perspectives (Stenros and Montola, 2012).Regardless of the setting and format, edu-larp emerges from these game-based leisure activities rather than the aforementioned professional contexts, offering the potential benefits of both spheres.While some larps feature a game-like structure with win conditions, others focus less upon concrete goals and more on the overall experience, which will vary from participant to participant.Not all larps are "games" in the sense of providing systems for success and failure.

32).
As edu-larp research is still developing, the majority of these sources emphasize theoretical principles regarding these benefits or feature anecdotal, observational reports from designers and educators.However, a few studies include some degree of data collection.Sancho, et al. (2009) collected grades and student evaluations after an online role-playing exercise used as a method to improve team work (2009).Similarly, in a language role-play study, Cheng (2009) used open-ended questions and quizzes regarding comprehension of persuasive techniques.Hou, et al. (2009) provide quantitative support for their claim that online role-playing improves cognitive skills such as motivation.Guenthner and Moore (2005) evaluated their leadership scenario with open-ended qualitative survey questions; responses indicated improvement in communication skills, depth of comprehension, and critical thinking.Howes and Cruz (2009) used a similar method to evaluate their scenario for science pedagogy, showing an increase in sensitivity to diversity and empathy.Harviainen, Lainema, and Saarinen (2012) also asked students open-ended questions to explore the limitations and impediments to a business simulation, finding issues with unrealistic trust, competitive play, and cheating.Brummel, et al. (2010) used a quantitative and qualitative mixed method design to evaluate their role-play on teaching responsible conduct of research, demonstrating increases in engagement and depth of understanding.Crow and Nelson (2014) also used a mixed method design with questionnaires and focus groups to examine the effectiveness of role-playing scenarios for training future coaches and physical education teachers, finding that students exhibited skills in the trained techniques and gained confidence.Each of these studies involved university or graduate level participants.
A few studies exist on edu-larp in secondary education.Mochocki (2014) evaluated edu-larp by assessing history subject matter retention over time in Polish high school students using a mixed methods design.A study by Harviainen and Savonsaari (2013) used teacher observations and reflective writing as a means to gather data on two edu-larps at the high school level in Finland: one focusing on secondary language acquisition, music history, and health education; the other emphasizing debate skills and empathy.Pitkänen (2014) used the stimulated recall method to study the effectiveness of historical edu-larp in producing empathy in students.In addition to increased empathy, the study found that students were intrinsically motivated to perform historical research by the scenario (Harviainen and Savonsaari, 2013).Pitkänen also attempted the stimulated recall method when studying how game design affects student thinking at Østerskov Efterskole, though further data gathering is needed for solid conclusions.Gjedde (2013) also studied students at Østerskov in a year-long mixed method study, investigating the edu-larp curriculum with regard to creative thinking linked with intrinsic motivation, as well as social creativity through participatory culture.Gjedde found that some students were motivated by the emotional, narrative elements of the game, whereas others found motivation in the strategic or competitive game properties.Additionally, the study reveals that students at Østerskov scored just as well on standardized tests as the national average in all subjects and slightly above average in specific areas (p.195).
A recent NCSBN National Simulation Study in the field of nursing simulation was conducted over two years, 10 schools, and 666 students (Hayden, et al., 2014).The study the examined actual competence and perceived competence of students in three groups: a control group that experienced traditional clinical hours and less than 10% simulation; a group that experienced 25% simulation in their curriculum; and a group that experienced 50% simulation.The investigators found no significant differences in test scores in any of the groups, which indicates that simulation education is at least as effective as traditional teaching in nursing.Pertinent to our current study, the group that received 50% simulation rated their performance as statistically higher than that of their peers and often reported feeling "very well prepared" for their profession, indicating higher self-confidence -i.e.perceived competence --than the other students in the study (p.S38).
Our case study contributes to this body of literature by providing exploratory quantitative and qualitative measurements of the impact of edu-larp on American disadvantaged middle school students along five dimensions of learning: intrinsic motivation, perceived competence, school engagement, team work, and leadership.To our knowledge, no mixed method studies exist that examine an edu-larp curriculum according to all of these measurements, though some of the above-mentioned studies examine individual dimensions.

METHOD 3.1 Participants
The study participants were economically disadvantaged students from a middle school class in a charter school located in innercity Los Angeles. 1 In the spring of 2013, the educational larp non-profit organization Seekers Unlimited (501c3) was invited to introduce edu-larp into the science curriculum in order to improve engagement.The primary investigator serves remotely from Austin, Texas, as Secretary on the Board of Directors for the organization.As Seekers Unlimited was offered the o p p o rt u n ity t o ru n e d u -l a rp s f o r a lo c a l c la s s f o r t h e e n t ire semester, the group of 24 students provided a convenient sample for research.The study was approved by the IRB for both Ashford University and the University of Texas at Dallas.The school involved in the study serves minority students, most of whom are impoverished by district standards, with 98% receiving free or reduced cost meals.While collecting demographic information regarding income level can prove difficult when studying underprivileged adolescent populations, the amount of free or reduced cost meals indicates the number of students whose families live between 130-185% below the poverty level (USDA, 2013).One quarter of the students at the study school are learning English as a second language.Students at the school perform well, scoring higher than the school district average on the California Standardized Test (CST) for science (see Table 1).Before data collection, the study was explained to students by a research assistant.Students were given the opportunity to give written and verbal assent and received consent forms for their parents to sign and return.Students were offered $10 gift cards to Wal-mart for participation at both time points.All but one of the students returned signed consent and assent forms and two students were absent from school at T2 for the quantitative surveys.Therefore, the collected research includes 23 out of 24 students with qualitative data for both time points and 21 students with both quantitative and qualitative data for T1 and T2.

Design
The investigators used a quasi-experimental, concurrent mixed methods design with parallel data collection on a convenient sample (see Figure 2).The primary investigator gathered observational, quantitative, and qualitative data at the beginning and the end of the semester.Collecting data with more than one method helps acquire richer, more complete data than a single method does (Tashakkori and Teddlie, 2003).
The project examined five overall dimensions of learning: perceived competence, intrinsic motivation, school engagement, leadership, and team work.The project also evaluated student interest and excitement for the edu-larp program.While the investigators planned to compare standardized achievement scores as well, the class' teacher failed to provide the student' final grades after the intervention, a problem other edu-larp researchers have faced (Mochocki 2014, in review).
Due to the single class of the small charter school, which only had one class per grade, no control group was available.Therefore, a comparison group was impossible and artificially establishing one --e.g.not allowing some children to participate in the larp intervention, which offered financial incentives and involved dedicated class time --would be unethical according to U.S. Institutional Review Board protocols.The mixed methods design of this study is grounded in a pragmatic paradigm, meaning that less emphasis is placed upon perfect experimental technique and more upon getting the most complete data possible within the given framework (Feilzer, 2010).Ideally, future iterations of this study should involve a control group.

Apparatus and Materials
Quantitative questions were entered into an online LimeSurvey, which students took as a group in the school's computer lab.In addition to collecting demographic information, the investigators utilized three psychometric surveys: the Intrinsic Motivation Inventory (IMI) (Ryan, 1982), the School Engagement Scale (SES) (Fredericks, et al., 2005), and the Student Self-Report Team work Assessment (SSRTA) (Zhuang, et al., 2008) Effort/Importance (ICC = .60),Interest/Enjoyment (ICC = .86)and for the overall scale (ICC = .70).The investigators also found that the IMI was temporally stable; e.g. the IMI is capable of measuring change over time (ICC = .92-.60).Reliability and validity data are not reported for the Value/Usefulness subscale, though the scale was used by previous researchers (Deci, et al., 1994).The SES was found to have good reliability ( = .63-.90) and demonstrated criterion-related validity (Fredericks, et al., 2011).Zhuang, et al., (2008) found good reliability for the three subscales of the SSRTA ( = .88, .80, and .78).Confirmatory factor analysis revealed that the 30-item SSRTA was a moderately good fit (CFI=.85).The variables were moderately to highly correlated (r =.59 -.79).
Qualitative data was collected by the primary investigator via individual semi-structured interviews and recorded on an MP3 player.The investigators designed the qualitative interview questions for both T1 and T2 (see Appendix A).Overall, the questions were structured to mirror the language and information of the quantitative surveys in order to compare the results from the two methods.The investigators also added questions to the T1 interviews to address outside factors such as interpersonal harmony in the classroom, engagement with media, and excitement level about the project.At T2, questions were added to assess specifically how edu-larp enhanced the learning experience Figure 2: Timeline for concurrent mixed methods design and student eagerness to try the method in the future (see Figure 5).As the qualitative questions were designed by the investigators for the purposes of this study, no tests for reliability with other items on the quantitative scales were possible, although some of the questions overlap.In addition, tests for reliability for ethnographic questions are not normative in qualitative research.Therefore, only the quantitative surveys have reliability information.
All data was kept on the investigators' computers and reviewed only by the investigators.

Procedure
Data was collected at two time points: in February 2013 before the intervention and in May 2013 after the addition of edu-larp to the science curriculum.Observational data was obtained by the primary investigator, who watched the teacher conduct the class for two hours at T1.The class took the quantitative surveys in unison using an online website accessed in the school's computer lab.The primary investigator conducted semi-structured qualitative interviews one-on-one in a mostly private study room at T1 and outside on a table in the schoolyard at T2.Therefore, the participants were in the presence of their peers for the surveys, but not the interviews.Their teacher was not present for any of the data collection.
Between T1 and T2, Seekers Unlimited piloted 8 free edu-larps over the course of a semester, each custom-designed to address specific concepts in the established science Core Curriculum.Each edu-larp scenario featured a different theme: 1) The Balloon Race, an exercise in which students competed in teams to move helium balloons around the classroom demonstrating velocity and forces; 2) Be Your Own Planet, in which students created stars, planets, moons, and comets in a solar system with bags and shredded paper in order to demonstrate density, mass, and volume; 3) The Great Phlogiston Debate, in which students played famous male and female scientists from the Enlightenment explaining their physical science and chemistry theories at a French salon; 4) Two larps called Element Hero, in which students created superheroes based on elements from the periodic table in order to fight an evil magician robot; 5) Noir, a murder investigation using acids, bases, and PH, in which students took turns playing detectives, suspects, and forensic scientists, ultimately culminating in a trial; 6) A student-teacher exercise in which the class taught each other scientific concepts from the Core Curriculum; 7) A Frankenstein-based "Monster Maker," in which students used organic Chemistry to create modeling clay monsters representing the six main chemical ingredients in life.Students could attack one another's monsters by successfully answering science questions devised by the class and approved by the teacher. 2   These edu-larps were designed to illustrate concepts on the upcoming CST exam through experiential means.Each edu-larp generally lasted one class period on one day of a school week (less than one hour), with supplemental instruction on these scientific concepts through traditional teaching and group work the rest of the week.Some of these larps were more game focused and some were more role-play focused.The designers introduced role-taking activities slowly over the course of the semester so students would become acclimated to the new format.While Be Your Own Planet featured some degree of personal identification with the planet as an abstract identity, students did not fully enact characters until the Great Phlogiston Debate.Some larps featured both roles and win conditions, such as superheroes defeating the magician in Element Hero by answering science questions.Other larps focused mostly upon immersion into the scenario, such as students playing scientists in a French salon describing their theories or playing teachers explaining scientific concepts to one another. 3  In order to avoid influencing the results of the data, the investigators knew little of the proposed science edu-larps at T1 except for possible genres.The investigators were not present for the delivery of the edu-larps, as they lived approx.2000 km away from the school at which the data was collected.Data was taken before the intervention began and after the larp curriculum was completed with little contact with the game designers regarding the content of the larps or specific student performance during the games.Similarly, the larp designers were not present at either data collection point.Qualitative and quantitative data were collected simultaneously, primarily due to the geographical distance between the data collection site and the researchers' homes.In order to integrate the two forms of data, the 2 Interview data only provided brief descriptions of each of the games in question.Full scenario descriptions and instructions are still in development by Seekers Unlimited for 2015 availability. 3The cost for each of these larps ranged from a few dollars for printing fees to a maximum of a hundred dollars for props or materials.These costs are significantly lower than most classroom games that require computer technology.
investigators planned to quantitize the qualitative interview data and perform a statistical analysis of the answers gleaned from interview questions (Sandelowski, Voils, and Knafl, 2009).
Therefore, the data presented in this study features student reactions from the end of the semester, not following each of the larps.The primary investigator also conducted 20-30 minute, semi-structured interviews with the class' teacher at T1 and T2, as well as a 1.5 hour interview with Aaron Vanek, one of the two larp designers, at T2.

Data Analysis
The In order to merge the qualitative and quantitative data, the investigators chose to quantitize the ethnographic results and contrast them with the quantitative results (Creswell, 2003;Teddlie and Tashakkori, 2003;Hayden, et al., 2014).The investigators both examined all participant interviews, summarizing responses to interview questions into one-or twoword responses, a relatively easy task as most of the young participants were quite terse.For each question, the investigators designated the range of responses on a 5 point scale: yes, mostly, sometimes, a little, and no.The number of responses were tallied and summarized in charts and bar graphs (Bowman and Standiford, in press), an example of which appears in Figure 4. Quantitized results were also examined for normal distribution, mean, standard deviation, percentages, and statistically tested using Student's t-test (see Table 6).Effect sizes for pairedsamples t-tests were not included, as sizes would appear overestimated for this type of analysis.
Additional spontaneous responses to semi-structured interview questions were organized according to themes and quantitized in a manner similar to previous thematic phenomenological work by the primary investigator (Bowman, 2013).For example, coding, organizing, and tallying data according to themes revealed that 18 out of 23 students spontaneously mentioned liking experiments in science at either T1 or T2, a significant finding considering the experiential learning aspect of edu-larp.For an extensive report on the ethnographic findings, see Bowman and Standiford (in press).

Demographic Characteristics
The class contained 14 Hispanic students, 8 black students, and 1 multiracial student (see Table 2).More than half of the students spoke Spanish and Spanish was a first language for 7 students.There were no significant differences in survey results with regard to age, gender, race/ethnicity, or language spoken (p > .05).

Observational Results
The primary investigator was only able to study the classroom during T1, as the students were taking their yearly standardized assessments during T2.At T1, the investigator witnessed a daily class meeting in which students were encouraged to express concerns pertaining to their school experience, which mainly consisted of complaints about uniform enforcement.The investigator also witnessed a student-developed skit enacting the American pioneer days for History, suggesting that larp-like activities were already present in the curriculum.
Overall, the students were respectful and engaged, but unruly, with the teacher having to ask several to leave the class for "time outs" for talking and off-task behavior.These behavioral issues were also prevalent during the quantitative data gathering in the school's computer lab.Both the teacher and Vanek, the edu-larp designer, described the class as particularly chaotic and unruly compared to other groups, with the teacher expressing the school's leniency for disruptive behavior as frustrating at T1.However, students displayed an overall respect for authority to the investigator, the teacher, and the principal during the observational period.The participants were cooperative and curious during the semi-structured interviews, particularly during T2, expressing a great deal of enthusiasm for the edu-larp intervention (see Figure 6).As the charter school requires an extensive application process, the investigators suspect that the student engagement with school and science is likely higher than average in their geographic area, as supported by the higher scores on the science assessments (see Table 1).
Quantitative results revealed no significant pre-post change for school engagement (see Table 5), team work, or the other intrinsic motivation subscales (effort/importance, value/usefulness, and perceived competence).In other words, only intrinsic motivation and interest/enjoyment increased from Time 1 to Time.F (1,21)=7.12,p < .05.At Time 2, the gender differences had equalized somewhat; the only area where boys continued to report higher levels was enjoyment of science, F(1, 19)= 5.90, p < .05.In summary, the larp intervention may have helped increase girls' confidence and may have enhanced their belief that science is useful to them.

Statistical Analysis of Quantitized Ethnographic Data
T-tests of the quantitized ethnographic data revealed significant change in perceived competence in science (t = -2.64,p < .05)(see Table 6).When the investigators selected only for students who self-rated as low performers (those who rated their perceived competence as less than 3), they found a greater prepost difference in perceived competence in science (n = 5, t = -3.50,p < .05)and perceived competence in school (n = 5, t = -5.30,p <.01).In other words, the interview data suggests that students believed their abilities in science had increased over the course of the semester, especially those for who previously believed themselves to have low competence.

Qualitative Results
As the semi-structured format is more flexible, the interviews allowed for a greater diversity of data, including specific responses regarding the edu-larp intervention.The qualitative portion of the study examined seven major dimensions at both time points: enjoyment of science, perceived competence in science, enjoyment of school, interpersonal harmony, team work, leadership, and interest in larp as a pedagogical method.Data taken at Time 1 also assessed student engagement with the media, gaming, and generic fiction.These results are reported in greater detail in Bowman and Standiford (in press).Several of the boys in the class regularly played video games together in a team self-dubbed "the Bros," indicating both active engagement in game-play and team work, important components of some edu-larp curricula.However, the girls in the class seemed mostly excluded from these activities, with one female reporting having difficulty interacting with boys due to their interest in games.

Data taken at
In terms of media genres, 5 students showed interest in detective stories in film, television, books, and games; 6 in superheroes; 6 in science fiction; 13 in fantasy; and 4 in history.These results indicate a student population fairly predisposed to engage with the content of the edu-larps with surprisingly little inherent resistance, possibly due to the rising popularity of video games and genre-based fiction aimed at a young adult audience.
The primary investigator further questioned the students regarding their enjoyment of science in an attempt to gauge their interest level.At Time 1, students in the case study showed a moderate degree of enjoyment, with 11 reporting in the unqualified affirmative.This number rose to 13 after the intervention at T2, with three students showing an increase.However, when asked if larp helped increase their interest, 19 students reported in the affirmative, indicating that the method itself was enjoyable for several of the students who otherwise felt ambivalent or disinterested in science (see Figure 5).
When queried about reasons for enjoying science at T1, 18 spontaneously reported favoring hands-on experiments: a high majority.5 described liking to learn how the world works, including building and taking apart items.3 described science as boring and 2 found science confusing.14 students disliked reading the textbook, 5 disliked writing, and 7 were frustrated sitting in class listening to lectures.These reports indicate a preference for active engagement with course material, suggesting a possible fit with the edu-larp approach, which includes a high degree of participation, student agency, and movement.
Students reported a strong enthusiasm for the larp method in the science classroom at T2.Three students expressed disliking science until the edu-larps, with 2 indicating that they only enjoy science while larping.8 students stated that larp made science more exciting, "fun," or interesting; similarly, 8 students indicated that the "fun" of learning while playing made the subject matter less "boring" and increased "attention." 2 students indicated a higher degree of comprehension of material and 4 appreciated the physical nature of larp, as opposed to sitting and taking notes.These findings correlate with the T1 preferences for experimental, hands-on activities; one student further indicated that the role-playing method offered an additional advantage to other classes that "just do experiments."Two students expressed previously disliking science, but now "liking" or "getting the hang of it."Only one student stated sometimes preferring to hear a lecture over playing a game, though the same student reported enjoyment of edu-larp in other areas of his interview.
The interviews revealed a large shift in perceived competence in science from T1 to T2 (see Figure 4).While several students expressed enjoyment of science at T1, a significantly lower number felt competent in the subject.At T2, the number of students expressing unqualified perceived competence jumped from 6 to 14. 13 believed that edu-larp as a method specifically increased their competence, with 4 more finding that edu-larp increased competence some of the time.Two students explained understanding the solar system, atoms, and density more thoroughly as a result of the larps.Other students described how larp helped them learn in a "different way" and increased comprehension.Some cited the role-playing, or "acting out," aspect as particularly helpful in aiding understanding.One student stated that larping had the students learning information above and beyond the necessary facts, such as how to investigate as a detective.Although in 6 of the interviews the primary investigator neglected to ask the follow-up question on larp helping increase competence, overall, most students reported that larp was helpful.The study also sought to measure the impact of edu-larp on student enjoyment of education and perceived competence in school as a whole.Overall, student sentiment toward school as a whole did not change dramatically over the course of the semester; about half the students reported enjoying school with an unqualified affirmative at T1 and only 7 reported an unqualified affirmative at T2, which transpired during their yearly standardized assessment.
When asked to share aspects of education they dislike at T1, 4 students mentioned the length of classes and 3 called school "boring" at times. 2 disliked "having a lot of work" and one found school stressful with his extracurricular activities.When asked about desired changes to school, 6 mentioned more physical education or playing outside. 2 described sitting still in class as difficult.Many of these complaints are common in the current traditional educational system.
Though these overall complaints did not change much after the intervention, 3 students mentioned school as a necessary means to improve their lives in the future.However, 16 of the students believed that larp increased their enjoyment of school when prompted.8 indicated that either they or their fellow students expressed excitement for edu-larp, with one female student sharing that she knew that students would have a "good day" when the class larped.One student stated that he previously "goofed off" in science before edu-larp.Another stated that in an exercise where she had to play a teacher educating the class, the larp encouraged her to "think" and "pay attention."Another student reported that the class was "hyped" up for days about the larps because the exercises were fun.
The study sought to examine the degree to which larps might affect the social dynamics of the classroom, including interpersonal harmony between peers and the development of prosocial skills such as team work and leadership.Overall, 13 students felt they got along well with their peers at T1, with the number raising to an unqualified 19 at T2. 13 students believed that edu-larp helped improve relations (see Figure 5), while several explained that group cohesion depended on the larp in question.6 stated that larp encouraged them to interact with everyone in the classroom, including students they normally would not, which helped them become better acquainted.One exception to larp's tendency to increase interpersonal harmony was an incident when a student was interrogated as a suspect by his peers in the detective scenario and became upset with the other students afterward.The investigators also evaluated the degree to which larp improved team work.The number of students who assessed themselves as good at team work rose from an unqualified 12 to 17 at the two time points, with 4 other students reporting "sometimes" and "mostly."Similarly, 17 felt that larp increased their team work, with 4 replying "sometimes."Some students reported that not all the games included team activities, though respondents mentioned 5 edu-larps as involving team work: the detective, monster, superhero, balloon race, and teaching games.Several students reported learning concepts from their peers and sharing ideas through group work, preferring interaction to learning from a book.Most of the students found learning in a group in a playful manner more "fun." The study further investigated leadership skills before and after the intervention.Some students showed leadership at T1. 3 of the students were involved in the ROTC at school teaching young cadets and 2 mentioned taking a leadership role in sports.5 felt they were role models to fellow students and were capable of influencing behavior.Several students disliked leadership, listing reasons such as discomfort speaking aloud, as well as difficulties getting other students to listen to them or take assignments seriously.
Student beliefs about their leadership skills remained roughly the same overall at T2, even though 13 believed that larp increased their abilities and 6 thought larp helped sometimes.4 students described playing a leader in the superhero game and 3 in the detective game.Other games that students cited involving leadership were the monster game in which students had to answer questions as a team to save their monsters, the larp in which students had to teach one another, and the Phlogiston Debate, where students played scientists convincing one another of their theories.One student reported encouraging students to work "instead of slack off."Others suggested that the teams worked together to solve problems.One student mentioned that his team members aided him in understanding scientific concepts that he previously did not comprehend.
Finally, student interest in larp as a whole was investigated at both time periods (see Figure 6).While not all students fully understood the concept of larp, most showed a high degree of interest in starting the method, with only 3 qualifying their enthusiasm.7 were excited by the novelty of the project and curious to try new methods.3 looked forward to interacting with students in new ways, 3 liked "hands-on" projects, and 4 were excited to engage in active participation rather than sitting and taking notes.
After the intervention, students showed the same degree of enthusiasm, with only 1 student preferring traditional class to larp.13 mentioned the activity as "fun," 3 found it "less boring," 6 were pleased with the interactivity, and 5 mentioned not having to read or take tests.6 students mentioned the ability to act in character and play a role as stimulating.Responses to the difficulty of the games varied; one student mentioned sometimes feeling confused or finding the material "hard," whereas another student called the exercises "easier" than normal class.One student mentioned not learning as much as a normal workbook assignment.Ultimately, all 23 students in the study expressed interest in trying edu-larp again in future classes, with one expressing regret that he will likely not experience edu-larp in high school.

DISCUSSION 5.1 Overview of the Findings
The investigators successfully gathered thorough data from students assessing their own development along five dimensions of learning before and after the semester-long edu-larp science program: perceived competence, intrinsic motivation, school engagement, team work, and leadership.The study also successfully acquired qualitative data from the students in the sample group regarding their overall experiences with the edularp method.
The quantitative data revealed an increase in overall intrinsic motivation and interest/enjoyment of science between the two time points.The qualitative and quantitative findings merged to reveal improvement in perceived competence in science.In ethnographic interviews, students expressed a strong belief that edu-larp aided in the development of all five dimensions, as well as interpersonal harmony and enjoyment of school.The students expressed excitement for the intervention and a unanimous interest in learning through edu-larp in the future.The data did not reveal significant changes in the areas of team work and leadership between the two time points, although several students offered examples in which these two factors were exercised during the course of the edu-larps.
Qualitatively, the participants expressed a high degree of interest in edu-larp.They also expressed that the method helped them enjoy and feel more competent in science as a whole.Several students mentioned that they were excited to come to class on larp days, with a couple of students mentioning they only liked science when paired with larp.Similarly, students rated their competence in science as having increased over the semester, with 17 students believing that edu-larp helped improved their abilities.Students discussed how acting out the concepts in science worked to increase their comprehension of material.
When comparing the two data sets, adding the edu-larp component to the existing science curriculum may have impacted the experience of students by increasing interest, engagement, and perceived competence in science through game play and role enactment.However, further research is needed to establish causation, as explained in the limitations section below.

Limitations
One limitation of the study was the lack of a comparison group, as the school was small and only one science class for that grade existed.This problem was unavoidable given the circumstances of the convenient sample.As a result, the small, non-random sample reduces the generalizability of the study, though researchers may find the data useful for further avenues of research.
After consulting with a statistician, the investigators determined that the study lacked a dependent variable, which necessitated multiple t-tests without the benefit of Bonferroni correction and put the results at risk for Type 1 error.However, since the study lacked a comparison group and the investigators uncovered no published studies that might suggest a possible dependent variable with this exploratory design, analytical options remained limited.Future studies should consider these limitations in their design and attempt to incorporate dependent variables and a control group.
In addition, though the teacher planned to take assessments of the students' abilities in science before and after the intervention, these assessments were not performed.The teacher also failed to supply grades from the current year, making a preand post-comparison of grades impossible.Therefore, data revealing the actual competence was not available to complement the findings on perceived competence.
Another limitation was that the T2 data was collected at the end of the semester rather than after the individual edu-larps.The investigators live in Texas rather than California and had limited time and funds.Therefore, they were not present to collect observational or behavioral data during the edu-larps themselves.The period of time between T1 and T2 was an entire semester featuring multiple activities, which made determining a causal connection between edu-larps and the quantitative measurements difficult.The ethnographic interviews helped address this issue by allowing students to directly express their perceptions of the edu-larp interventions in question, though the amount of time elapsed between the larps and the interviews may have affected student responses.
Disagreement existed in the data between the two methods in some cases.For example, the perceived competence in science result was non-significant from survey data (p=.119),but was significant in the quantitized interview data (p=.015).The methods also disagreed in terms of their ability to measure edularp as a specific factor.The quantitative questions measured interest in science overall, which students may or may not have attributed to the edu-larp curriculum.Alternately, the ethnographic questions specifically asked students to assess edularp as a method by asking additional questions that allowed for semi-structured responses.Therefore, more positive findings in all areas were revealed in the qualitative interviews, perhaps as the result of the student excitement for game play in the classroom rather than general improvement.Some questions were repeated in the quantitative and qualitative surveys, such as "Do you enjoy science?"The investigators found this extra data taken in another manner useful, as the quantitative survey was quite lengthy.The primary investigator observed fatigue and distraction during the 94 question survey.These factors may have contributed to the poor reliability of the School Engagement Scale in the sample.
Finally, as with any study measuring the perception of subjects who have knowledge of the intervention in question, potential bias in the responses is possible, a phenomenon known as the Rosenthal effect.For example, students may have responded in the affirmative to questions about larp increasing their perceived competency in science due to their knowledge that the intervention was designed to facilitate this increase.Also, students may respond positively due to the direct questioning by the investigator, who they perceive to be an authority figure.While these limitations are problems inherent to many studies involving participant responses, the investigators trust that the subjects responded truthfully when prompted.Additionally, the investigators were not involved with the intervention itself, in part so that students would perceive this detachment as more neutral.Future studies could augment the participant-response method with behavioral measures (Magee, 2014), though such a design would further increase the complexity of the study.
Ultimately, while the quantitative results are limited due to the lack of the control group and small sample size, the qualitative results strengthen these conclusions by featuring questions specifically geared toward the edu-larp pedagogy.Therefore, the conclusions of this pilot study are not generalizable, but should serve as a useful starting point for future edu-larp research.

Findings Compared with the Literature
Overall, the participants were already highly engaged with school and interested in science before the intervention.This finding is probably due to the fact that the charter school format requires an application process and screens applicants for school engagement and achievement.Students reported a high degree of interest in "experiments" in science at T1, as well as some interest in generic fiction and games.Thus, the group was likely to find the edu-larp format appealing, as indicated by their initial enthusiasm to engage in the project and their excitement for larp after the intervention.
The major finding of the quantitative surveys was that larp significantly improved enjoyment of and intrinsic motivation for science.These results harmonize well with the qualitative findings.This data supports the notion present in much of the literature suggesting that edu-larp promotes intrinsic motivation through active engagement with the material and a desire to perform well in the scenario (Larsson 2004 Henriksen (2006) emphasizes that an excess of "fun," might disrupt the learning process, asserting that frustration should be the primary driver by utilizing the desires and needs of the participants in the embedded structure of game play (p.12).
Therefore, if edu-larp tends to encourage intrinsic motivation, such motivation must include curriculum-related outcomes within the structure to remain effective.
The mixed method approach allowed the investigators' to discover data in multiple ways, a benefit not available in a single-method study.Quantitative data alone would not have revealed increases in perceived competence, but the extensive interviews and quantitized qualitative data offered additional insights.This finding resonates with edu-larp theory regarding self-efficacy and perceived competence.As Balzac (2011) explains, the edu-larp process can enhance student feelings of self-efficacy through setting and achieving goals, a concept closely related to perceived competence.Larp youth camp organizer Mark Hoge further describes how games often encourage participants to share their unique skills and perspectives in order to increase the success of the group as a whole (Hoge 2013, 48-49).This sort of empowerment is not always afforded to young people, who often rely on adults to make decisions for them or to force them into a course of action (Blatner, 2009).This finding also resonates with the NCSBN National Simulation Study on nursing, which revealed that students with more experience in simulation rated their confidence in their abilities as higher than their peers did (Hayden, et al., 2014).
Another interesting finding was a shift in gender differences from T1 to T2.Quantitative data taken at T1 revealed that boys reported significantly greater enjoyment, intrinsic motivation, behavioral engagement, and value/usefulness in science.In addition, the boys reported higher perceived competence in school than girls.By T2, these gender differences had equalized somewhat, as the only area where boys continued to report higher levels was enjoyment of science.This finding connects with the ethnographic data regarding the gender differences, with many of the boys involved in a video gaming group called the "Bros" that did not include female students and several of the girls reporting disinterest in games.By T2, the girls in the class reported enjoying the edu-larp format, indicating a greater inclusivity in game activities in the sample group after the intervention.
Statistically, no significant increases in team work and leadership were revealed from T1 to T2.This finding may result from the fact that several students initially reported competence in these areas.Additionally, the larps were not specifically structured to improve leadership and team work in the way that other scenarios are (see Guenthner and Moore, 2005).In order to measure improvement in these dimensions, the investigators' suggest rigorously designing edu-larps that encourage students to take turns in leadership roles and work together in teams to solve problems.Examining past scenarios and assessments in the fields of edu-larp, simulation, and educational drama may help further develop edu-larp curriculum and establish measurable outcomes in these areas.
While the data did not suggest strong increases in overall team work abilities, the semi-structured interviews revealed that the collaborative nature of the group work in the edu-larp classroom helped students learn from one another and get along better (Figure 5).Students appreciated the ability to interact with their peers in new ways, particularly with individuals outside their normal friend groups.One exception to larp's ability to increase interpersonal harmony was an incident when a student was interrogated as a suspect by his peers in the detective scenario and became upset with the other students afterward.This participant probably experienced a sense of bleed, where in-game emotions affected his out-of-character feelings toward others (Montola, 2010;Bowman, 2013).Thus, facilitators should pay close attention to classroom power dynamics when assigning potentially sensitive roles and remain aware of potentially triggering situations (Kessock, 2013;Brown, 2014).Harviainen and Savonsaari (2013) suggest that teachers should only implement highly emotional edu-larps if they know the classroom dynamics well.
Similarly, regardless of the statistical findings, several students did report adopting a leadership role in the scenarios, with a few indicating that their leadership led their group to victory in the games.In any given classroom, not all students will exhibit a preference for leadership or facility with the skill.Also, designing edu-larp scenarios that allow students the opportunity for equivalent leadership roles is difficult.However, edu-larp does provide a unique environment in which students may occupy roles that would they would not otherwise find available to them, such as the hostess of a French salon in the eighteenth century or a detective in a murder investigation.For example, one student shared, "We did [the] detective game and I actually felt like I was actually a detective.*laughs* And then we did a superhero and I had. . .a really awesome power, so I did feel like a leader."Another student expressed great enthusiasm for the investigation game, stating, "I told my mom when . . .I was five years old, I said 'I want to be a detective.'So, once I got to [play] it at school, I was like, 'No way.I'm finally a detective!'"Vanek further explained that one of the students had asked specifically for a detective scenario; therefore, Noir was designed according to student demand.The empowerment involved in the agency afforded by such roles, whether leadership or otherwise, is considerable.
Ultimately, while further research is needed to investigate the generalizability of the findings of this case study, the data indicates strong potential for edu-larp as a method of classroom engagement.These results resonate with learning theories that posit experiential learning (Kolb, 1984) and situated learning (Lave and Wenger, 1991) as beneficial to knowledge acquisition and training of expertise (Dreyfus and Dreyfus, 1980).
However, the efficacy of any program depends upon the success of facilitation on the part of the designer and/or teachers; in this case study, the designers were present for most of the scenarios and the teacher indicated a high degree of willingness.The teacher, who had never previously larped, now plans to use the curriculum in future classes without the designers present and is adding her own tweaks to the sy stems in order to adjust for classroom dynamics.
As with any innovation, no one method should be viewed as a "cure all" for education's problems.However, this study indicates that edu-larp offers opportunities for greater motivation, engagement, interaction with peers, collaboration, and comprehension of material.

Figure 3 :
Figure 3: Quantitative pre-post differences in intrinsic motivation and interest/enjoyment by measuring scale mean scores

Figure 4 :
Figure 4: Perceived competence in science at Time 1 and Time 2.

Figure 5 :
Figure 5: Belief that edu-larp increased enjoyment of science, competence in science, enjoyment of school, interpersonal harmony, team work, and leadership at Time 2.

Figure 6 :
Figure 6: Interest/enjoyment of the edu-larp method at Time 1 and Time 2.

Table 1 :
School statistics of the study school vs. the Los Angeles Unified School District (LAUSD) as a whole King, 2011)rs fully transcribed the qualitative interviews and coded them thematically (Braun and Clarke, 2006;King, 2011).Quantitative data was analyzed using SPSS version 22 to calculate descriptive statistics and test for pre-post differences using paired samples t-tests and one-way analysis of variance (ANOVA).Due to the one-group design of this study, the best statistical test was the Student's t-test.The investigators did, however, divide the group by gender and ran ANOVA to examine variables.After consulting with a statistician, the investigators determined that the lack of dependent variable for this study, which necessitated multiple t-tests without the benefit of Bonferroni correction, puts the results at risk for Type 1 error.
However, since the study lacked a comparison group and the investigators uncovered no published studies that might suggest a possible dependent variable with this exploratory design, analytical options remained limited.Future studies should consider this limitation in their design and attempt to incorporate a dependent variable and control group.

Table 2 :
Demographic characteristics of the sample

Table 3 :
Time 1 to Time 2 comparison of Intrinsic Motivation, School Engagement, Self-Reported Team work scales.

Table 4 :
Time 1 to Time 2 comparison of Intrinsic Motivation subscales.

Table 5 :
Time 1 to Time 2 comparison of School Engagement subscales.
Time 1 assessed student engagement with the media, particularly television, film, books, and games.These questions were intended to gauge whether the population showed a predisposition toward game play and toward the genres explored by the upcoming larps, such as detectives, superheroes, science fiction, fantasy, and history.Only one student reported having tried analog role-playing, having played a session of Dungeons & Dragons before.Several students showed a predisposition toward game play, with 15 out of 23 enjoying games and 5 enjoying only some games.Students reported video games, board games, sports, and other physical play activities in this general category.Video games listed included mainly firstperson shooters, racing games, building simulators, and some RPGs.

Table 6 :
Mixed Methods (Quantitized) Results: T1-T2 comparison of perceived school competence and perceived science competence