Index:
From the thematic data, three major knowledge claims were made:
Instructors were involved more formally as revisers of the curriculum by having voluntary weekly input solicited in the form of electronic mail messages. The author prepared for large-scale formal interviewing of collaborating students by obtaining the technical means of recording and transcribing the interview data. A video camera was obtained; video mount points were established in the lab; microphones were installed to augment current audio recording practice; cabling for video-audio recorder interconnection and a transcription machine were all obtained. Trial and evaluation of the video and audiotape data acquisition was piloted for data collection for later research.
This study attempted to improve the quality of the educational experience of the many student in Physics 152 Laboratory (PHYS 152L) through reflections upon personal actions. Students, instructors and curricular designers of PHYS 152L attempted to improve the rationality and justice of their own practices, their understandings of these practices and the situations in which these practices are carried out. Collected observations and their subsequent interpretations by both the author and the student-collaborators were used to address the guiding questions.
These interviews with participants were conducted in as open-ended a fashion as readily achievable. Interviews started with vague inquiries like "What did you think of..." or "Tell me about..." or similar cues designed to get participants talking about their major perceptions of the activities. Participant comments were first listed in the field notes until participants exhausted their memory on the topic. Next, the meanings of each comment listed in the fieldnotes were probed by requesting supplemental information, and the in-depth descriptions added to the notes. Then the entire activity was reviewed, part by part and question by question, and comments were elicited for each portion of the activity. Finally, summary comments on the activity and advice for improving the activity by enlarging activities and cutting others was solicited, along with advice to be given to hypothetical 'other students' who would be taking the course in future semesters.
I kept extensive field notes of the conversations. These field notes were the primary criteria used to decide which sections of the interview audiotape would be transcribed. All of the fieldnotes (over 500 pages) were closely examined and reflected upon, then critical segments of audiotape were closely examined and transcribed (about 10% of the total). Over 200 hours of audiotapes interviews were collected.
Approximately 35 hours of videotape were collected (all of which were replicated by the audiotaped data) of participants completing the curricular experiments and attempting to 'think aloud' or describe aloud what they were doing and thinking about. These also were not transcribed in total; selected segments were transcribed and were used to annotate fieldnote transcriptions of appropriate data exemplars. For instance, if participants repeatedly described some portion of an activity as confusing during interviews, this was first noted in the fieldnotes, then transcribed from audiotape and finally annotated by partial videotape transcription describing the participant behavior and apparatus interaction in question.
Other artifacts collected in this study included copies of all participants' lab reports from Physics 152L and some from similar courses, participant annotated laboratory manuals and other curricula materials and copies of e-mail correspondence between participants and the author.
These different data collection techniques repeated across a number of students complemented one another by examining laboratory practice from a wide variety of viewpoints and providing a means of triangulation to validate or disprove assertions drawn from the data. Two GTA interviews were conducted at the end of the semester and at the halfway point during the semester, and these were also used for the pruposes of triangulation. Field note data was treated as the primary source for student commentary, and the primary source for observations on participant behavior in the laboratory.
I commenced the data analysis by fastening copies of all field notes and some transcripts and laboratory report extracts to the walls of two large rooms. Here approximately 550 8.5" x 11" sheets of data could be viewed with ease simultaneously across both different students and curricular activities. These data were posted for several weeks (some for months). I examined them repeatedly over time and reflected upon them at length. My analysis of these data started by examining, annotating and highlighting data patterns (perceived similarities) using different highlighter and marker color codes, colored post-it stickers and marker symbols. My analysis proceeded by preparing summaries and concept maps from these data patterns.
Next, I returned to secondary data sources (videotapes, additional transcriptions, participant commentary on the summaries) for re-examination or additional transcription as seemed appropriate to clarify the patterns. From these originally quite vague data, I generated a number of specific categories by repeatedly coding and recoding, concept-mapping and summarizing the data. I further refined the categories by identifying characteristic properties and discriminating criteria for each and by choosing high-quality exemplars to illustrate each category. These categories emerged naturally from the interaction of the data, the researcher and the participants in the study.
I documented these categories, and used them to make highly situational and specific knowledge claims about various curricular activities. Next, I thematically grouped these categories and used them to formulate general assertions or knowledge claims concerning the curriculum as a whole. These knowledge claims would be made in such a manner as to guide generalized curriculum development. I formulated these assertions upon their enactability -- the principle value of these assertions lay in their ability to guide curricular interventions -- to determine specific, active curricular reforms that could be used to modify and better inform curricular practice in the laboratory.
Finally, these knowledge claims were documented with categorical evidence and were presented in writing to several the participants in the study. The participants read, reflected and commented upon the appropriateness of these claims, providing validations or alternate interpretations. The claims were then used along with exerpts from Physics education research to restate the major curricular goals for Physics 152L, which were also presented from commentary to the study participants.
As a result of this study, the PHYS 152L curriculum is being significantly reformed and the new curriculum (now in preparation) should be in place for student enrollment in Summer of 1995. This reformed curriculum will feature a number of changes as follows:
1. The reformed curriculum will be rewritten to portray the importance of report writing, measurement analysis and cooperative learning and reporting. These will be made overt goals of the activities. Initially, one assignment will become a mandatory (not optional) group activity, and more may be similarly designated after trial. The enaction of these changes to the curriculum will have to be assessed through additional critical examination via action research.
2. To aid assessment and to continue technical refinement, curricular evaluation through student observation and student and instructor feedback will be entrenched into regular practice. This will be attempted by creating a support framework so that each instructional staff member carries out some limited form of critical inquiry during the semester. PHYS 152L routinely has between twenty and forty instructional staff members. Many of these have been informally evaluating and reporting curricular shortcomings to be addressed by the author for several semesters. This process will be formalized by requiring it in all instructional job descriptions, and by offering informal encouragement, extra credit or additional employment opportunities for those interested in in-depth curricular reform and development. The formal separation now extant between curricular development personnel (the development crew) and instructional personnel (the teaching staff) will be reduced or eliminated by allowing course staff to freely move between both kinds of activity. This will also create an atmosphere of critical evaluation which models the rational investigation conducted by laboratory scientists and professionals for the benefit of the students in the course.
3. Formal instruction in the use of computer data analysis and presentation software will be incorporated into the new curriculum and use of these tools will become integral to all activities. They will become required, not optional events. This will likely be part of the very first activity students complete in their lab.
4. The reformed curriculum will be more closely integrated with the lecture through topical matching and assessment. More experiments will be added to the curriculum and these activities will contain appropriate illustrative activities upon the topics of momentum conservation and a much more general and kinesthetic activity on rotational motion. Apparatus will be more varied than in the previous curriculum. PHYS 152L will become a six experiment course (one every two weeks), instead of the current four (one every three weeks), allowing tighter synchronization. This will fully utilize all space and instructional resources available to the course for the foreseeable future. Questions typical of laboratory practice will be included in the pool of items from which midterm and final exam questions are drawn for the whole PHYS 152 course.
5. Laboratory assessment will occur in a more timely fashion. Prelaboratory questions in the reformed curriculum will be graded during the first thirty minutes of class data collection and will then be immediately returned to all students. This will require that the PLQs be reduced in length, and will likely mean that the PLQs will not be fully graded -- collected PLQ sets will be spot checked (closely graded on a chosen subset of specific questions) and the remainder simply scanned for completeness. Students thus will have their PLQs when collecting, interpreting and reporting their laboratory data.
6. The amount of equation verification during activities in the reformed curriculum will be reduced, and more interpretive, non-numeric activities will be assigned.
The categories, knowledge claims and recommendations of this study have been primarily generated to address the specific needs of Physics 152 Laboratory curricular reform. This is an appropriate and sufficient task in its own right, given that some ten thousand students have participated in these activities in the last five years and a similar number will directly benefit from this study in the next five years. However, there are a number of implications from this research that hold significant bearing and possible impact upon the Physics Education and Science Education communities in general, particularly upon introductory university instructional laboratories.
1. Laboratory course goals should include the deliberate, explicit student acquisition of skills and techniques required by working engineers and scientists. These include reporting skills; modern (e.g., computer aided) data acquisition; computer data presentation, reduction and analysis; the use of measurement analysis and statistical and graphical analysis; and critical, reflective analysis methods. These activities should be practiced for mastery (e.g., by completing several similar format laboratory reports) in the contexts provided by a variety of appropriate illustrative phenomena for the laboratory subject being taught (e.g. mechanics, biology, chemistry). Student attention should be explicitly focused upon the acquisition of these skills to promote their own mastery learning and develop student motivation through awareness of and confidence in their own newly acquired abilities. As well, the activities should be chosen to well-illustrate the lecture material.
2. Action research and critical theory provide appropriate paradigms to guide reflective practice and the involvement of all participants in the learning science laboratory. The tenets of such critical curricular development appropriately reflect and guide the spirit of analytic and critical examination of both laboratory phenomena and day to day instructional practice. Action research provides a means for all participants to contribute in a rational manner to learning practice, and takes a profoundly nontraditional attitude towards the role of Instructors, Teaching Assistants and Graders -- encouraging these people to become curricular assessors and by extension, curriculum developers. This means that all participants in the laboratory have a responsibility towards worthwhile and just practice, not only the Professor and course developers. It also suggests that a primary responsibility of all Teaching Assistants is course assessment and development.
Action research encourages appropriate TA preparation as professional educators in their respective fields by requiring them to become active in interpreting their fields of study and in examining the learning of their students. This refutes the widely held notion that TAs and even science faculty must await professional education researchers to provide such initiatives in educational settings. Some elementary training in action research methodology and critical theory should be included in the background of all professional educators, particularly those in the sciences.
3. Action research provides an ideal means of sustaining a commitment to curricular change and refinement, by allowing many participants in the educational setting to contribute. The sum total of these contributions is far greater than any single curriculum developer's efforts are likely to be by simple additive power, and greatly supplement the motivation and drive required to sustain a commitment to improvement. While educational research and trained educational researchers can provide profound insights and interpretive ability to curriculum development, without this essential spirit of rational inquiry in many participants in each educational setting such efforts are unlikely to be understood, productive or continued.
Dan MacIsaac, 1996 (http://www.physics.nau.edu/~danmac)