«DESIGN-BASED RESEARCH IN PHYSICS EDUCATION: A REVIEW* Richard R. Hake In this chapter I argue that some physics education research (PER) is ...»
DESIGN-BASED RESEARCH IN PHYSICS EDUCATION: A REVIEW*
Richard R. Hake
In this chapter I argue that some physics education research (PER) is design-based
research (DBR) and that an important DBR-like facet of PER, the pre/post testing
movement, has the potential to improve drastically the effectiveness of undergraduate
instruction generally, the education of pre-service teachers in particular, and, as a net
result, the education of the general population.
I. Some Physics Education Research Is Design-Based Research In their resource letter on physics education research, McDermott & Redish (1999) list about 160 empirical studies, extending over almost three decades, that (a) focus on the learning of physics by students, (b) represent systematic research, and (c) give procedures in sufficient detail that they can be reproduced. My own effort in developing, testing, and disseminating Socratic dialogue inducing (SDI) laboratories is rather typical of the work reported by long-established physics education research groups and referenced by McDermott & Redish.
SDI laboratories emphasize hands- and heads-on experience with simple mechanics experiments and facilitate the interactive engagement of students with course material. They are designed to
promote students’ mental construction of concepts through:
(1) interactive engagement of students who are induced to think constructively about simple Newtonian experiments which produce conflict with their commonsense understandings;
(2) the Socratic method (e.g., Arons, 1997; Hake, 1992, 2002d) of the historical Socrates (Vlastos, 1990, 1991), not Plato’s alter ego in the Meno (as mistakenly assumed by many - even some physicists), utilized by experienced instructors who have a good understanding of the material and are aware of common student preconceptions and failings;
(3) considerable interaction between students and instructors and thus a degree of individualized instruction;
(4) extensive use of multiple representations (verbal, written, pictorial, diagrammatic, graphical, and mathematical) to model physical systems;
(5) real world situations and kinesthetic sensations (which promote student interest and intensify cognitive conflict when students’ direct sensory experience does not conform to their conceptions);
(6) cooperative group effort and peer discussions;
(7) repeated exposure to the coherent Newtonian explanation in many different contexts.
*The reference is: Hake, R.R. 2007. "Design-Based Research in Physics Education Research: A Review," in A.E.
Kelly, R.A. Lesh, & J.Y. Baek, eds. (in press), Handbook of Design Research Methods in Mathematics, Science, and Technology Education.Erlbaum; online at http://www.physics.indiana.edu/~hake/DBR-Physics3.pdf.
I welcome comments and suggestions addressed to firstname.lastname@example.org.
Partially supported by NSF Grant DUE/MDR-9253965.
© Richard R. Hake, 7 March 2007.
As described in Hake (1987, 1991, 1992, 2000, 2002d, 2007c), Hake & Wakeland (1997), and Tobias & Hake (1988), SDI laboratories were inspired by the astute empirical observations of Arnold Arons (1973, 1974, 1983, 1986, 1997) who had the uncommon sense to “shut up and listen to what students say” in response to probing Socratic questions.
In numerous publications, I scientifically and iteratively developed (Hake, 1987), explored (Tobias & Hake, 1988; Hake, 1991; 1992), confirmed (Hake, 1998a, 1998b, 2002a, 2002b, 2005, 2006, 2007a), and disseminated (Hake, 2000, 2002b, 2002d, 2007b) SDI laboratories. My research and development involved active innovation and intervention in the classrooms of introductory physics classes for prospective elementary teachers (Hake, 1991), premedical students (Hake, 1987, 1992; Hake & Wakeland, 1997), and even nonphysical science professors (Tobias & Hake, 1988). Further, my research and development drew upon models from design and engineering, in that SDI laboratories were designed initially by taking into account my own teaching experience, the advice of the late Arnold Arons, and the physics education and cognitive science literature. Then, trial runs that exposed design failures and successes were carried out in regularly scheduled courses; this phase was followed by exploratory out-of-class research with paid student subjects involving videotape analysis of SDI laboratory sessions (Hake, 2000) and interviews with students.
Three redesigns, retests, and more exploratory, in- and out-of-class research and development over many cycles of application—all in typical engineering fashion—generated new ideas for physics teaching (Hake, 1987, 1992, 2007b; Tobias & Hake, 1988; Hake & Wakeland, 1997) and contributed to the transformation of the traditional recipe laboratory. I sought to understand learning and teaching while I was active as the instructor (Hake, 1987, 1992; Tobias & Hake, 1988; Hake & Wakeland, 1997). As explained in Hake (2002a) (in the section titled “Can Educational Research Be Scientific Research?”), my research and development were examples of use-inspired, basic scientific research, consistent with the theses of Shavelson and Towne (2002) and Stokes (1997). Such work contributed to the movement of at least some introductory mechanics courses from malfunction to function, as shown by pre/post test results (Hake, 1998a, 1998b; 2002a, 2002b, 2005, 2006, 2007a).
Considering the above two paragraphs, I submit that some PER qualifies as design-based research as characterized by Kelly (2003a).
BUT WAIT! Should not the major concern of education research be K-12, as appears to be the area of activity for most education specialists, psychologists, and cognitive scientists? Not necessarily. The NSF’s (1996) report Shaping the Future hit the nail squarely on the head (my
Many faculty in Science, Mathematics, Engineering, & Technology (SME&T) at the postsecondary level continue to blame the schools for sending underprepared students to them. But, increasingly the higher education community has come to recognize the fact that teachers and principals in the K-12 system are all people who have been educated at the undergraduate level, mostly in situations in which SME&T programs have not taken seriously enough their vital part of the responsibility for the quality of America’s teachers.
2 Hake 3
In my opinion, the DBR-like, pre/post testing movement, stimulated to some extent by physics education research, has the potential to improve undergraduate science instruction dramatically and thereby upgrade K–12 science education. Currently, prospective K–12 teachers derive little conceptual understanding from traditional, undergraduate, introductory science courses; then they tend to teach as they were taught, with similar negative results.
As emphasized by Goodlad (1990):
Few matters are more important than the quality of the teachers in our nation's schools. Few matters are as neglected....A central thesis of this book is that there is a natural connection between good teachers and good schools and that this connection has been largely ignored....It is folly to assume that schools can be exemplary when their stewards are illprepared. (My italics, pp. xi-xii.) II. Pre/post Testing In Physics Education Research The pre/post testing movement in PER was initiated by the landmark work of Ibrahim Halloun and David Hestenes (1985a, 1985b). Previously, in “Lessons from the Physics Education Reform
Effort” (Hake, 2002a) I wrote:
For over three decades, physics education researchers repeatedly showed that traditional (T) introductory physics courses with passive-student lectures, recipe laboratories, and algorithmic problem exams were of limited value in enhancing conceptual understanding of the subject (McDermott & Redish, 1999). Unfortunately, this work was largely ignored by the physics and education communities until Halloun and Hestenes (1985a, 1985b) devised the Mechanics Diagnostic (MD) test of conceptual understanding of Newtonian mechanics.
Among the virtues of the MD, and the subsequent Force Concept Inventory (FCI) tests (Hestenes, Wells, & Swackhamer, 1992) are: (a) the multiple-choice format facilitates relatively easy administration of the tests to thousands of students, and (b) the questions probe for conceptual understanding of basic concepts of Newtonian mechanics in a way that is understandable to the novice who has never taken a physics course (and thus can be given as an introductory-course pretest), while at the same time are rigorous enough for the initiate.
Construction of the MD test involved laborious qualitative analysis of extensive interviews with students and the study of prior qualitative and quantitative work on misconceptions (McDermott & Redish, 1999). All this led to a “taxonomy of common sense concepts about motion” (Halloun & Hestenes, 1985b; Hestenes, Wells, & Swackhamer, 1992) and finally the formulation of a balanced and valid test that has proven consistently to be highly reliable, as judged by relatively high Kuder–Richardson reliability coefficients KR-20 in the 0.8 to 0.9 range (see, e.g., Halloun & Hestenes, 1985b; Hake, 1998a, 1998b).
3 Hake 4
Halloun and Hestenes (1985a, 1985b) then used the MD in quantitative classroom research involving massive pre/post testing of students in both calculus and noncalculus-based
introductory physics courses at Arizona State University. Their conclusions were:
1.... the student’s initial, qualitative, common-sense beliefs about motion and...[its]..
.causes have a large effect on performance in physics, but conventional instruction induces only a small change in those beliefs.
2. Considering the wide differences in the teaching styles of the four professors...[involved in the study]...the basic knowledge gain under conventional instruction is essentially independent of the professor. [pp. 1048 of Halloun & Hestenes (1985a)] BUT WAIT! Can multiple choice tests gauge higher level cognitive outcomes such as the conceptual understanding of Newtonian mechanics? Wilson & Bertenthal (2005) think so,
writing (p. 94):
Performance assessment is an approach that offers great potential for assessing complex thinking and learning abilities, but multiple choice items also have their strengths. For example, although many people recognize that multiple-choice items are an efficient and effective way of determining how well students have acquired basic content knowledge, many do not recognize that they can also be used to measure complex cognitive processes.
For example, the Force Concept Inventory... [Hestenes, Wells, & Swackhamer, 1992]...
is an assessment that uses multiple-choice items to tap into higher-level cognitive processes.
The Halloun & Hestenes (1985a, 1985b) research results were consistent with the findings of many researchers in physics education (McDermott & Redish, 1999), which suggested that traditional, passive-student, introductory physics courses, even those delivered by the most talented and popular instructors, imparted little conceptual understanding of Newtonian mechanics. But the Halloun and Hestenes research went far beyond earlier work because it offered physics teachers and researchers a valid and consistently reliable test that could be employed to gauge the effectiveness of traditional mechanics instruction, then to track continually the merit of the nontraditional methods with respect to (a) traditional methods, (b) one another, and (c) various modes of implementation. Thus, it could contribute to a steady, albeit very slow, iterative increase in the effectiveness of introductory mechanics instruction nationwide.
For example, consider the MD/FCI-induced changes in introductory physics courses at pacesetting Harvard University and the Massachusetts Institute of Technology. Harvard
University’s Mazur (1997) wrote (p. 4):
When reading this [Halloun & Hestenes, (1985a, 1985b, 1987); Hestenes, 1987],... my first reaction was “Not my students...!” Intrigued, I decided to test my own students’ conceptual understanding, as well as that of physics majors at Harvard....the results of the test came as a shock: The students fared hardly better on the Halloun and Hestenes test [1985a] than on their midterm exam. Yet the Halloun and Hestenes test is simple, whereas the material covered by the examination (rotational dynamics, moments of inertia) is of far greater difficulty, or so I thought.
4 Hake 5
In Table I, p. 972, of Crouch & Mazur (2001), note:
1. The abrupt increase in the average normalized gain g (see below) from 0.25 in 1990 to
0.49 in 1991 when Mazur replaced his passive-student lectures (that netted very positive student evaluations—many administrators erroneously regard student evaluations as valid measures of students’ learning!)—with the interactive engagement of peer instruction.
2. The gradual increase in the average normalized gain g from 0.49 in 1991 to 0.74 in 1997 as various improvements (Crouch & Mazur, 2001) were made in the implementation of peer instruction.
MIT’s John Belcher (2003), describing his institute’s introductory physics course transition from
traditional to interactive-engagement, wrote (p. 8):
What is the motivation for this transition to such a different mode for teaching introductory physics? First, the traditional lecture/recitation format for teaching 8.01 and 8.02 has had a 40–50% attendance rate, even with spectacularly good lecturers (e.g., Professor Walter Lewin), and a 10% or higher failure rate. Second, there has been a range of educational innovations at universities other than MIT over the last few decades that demonstrate that any pedagogy using “interactive-engagement” methods results in higher learning gains as compared to the traditional lecture format (e.g., see Halloun & Hestenes, 1985a, 1985b;
Hake, 1998a; Crouch & Mazur, 2001), usually accompanied by lower failure rates. Finally, the mainline introductory physics courses at MIT do not have a laboratory component.