Categories from the 4th Cycle of AR

Characteristic 1: Participant Learning Claims

Study participants claim that they are acquiring a series of previously unfamiliar technical skills as a result of their PHYS 152L experiences. Skills reported by participants in interviews include proficiency in formal reporting, laboratory data acquisition, measurement analysis techniques, computer graph generation and interpretation and least squares fitting practice. Furthermore, participants described their acquisition of these skills with some considerable pride and indicated that they felt the development of these skills was appropriate and worthwhile practice for the PHYS 152 course, their other undergraduate studies, and probably in their coming professional engineering careers. These are the kinds of things that they felt engineers do and that they should be learning.

The perception of student proficiency is characteristic of every curricular activity summarized in the data set and became strongly stated by student participants as the study progressed, after student participants completed a number of activities, reflected upon their experiences, modified their practice, and obtained feedback from their instructors through graded reports and MAs. This claim to learned skills was most evident in the prevalence of the things participants claimed they learned in PHYS 152L during their final interviews. By order of prevalence, participants felt that the course had reinforced the lecture material; taught report writing skills; taught graphing, least squares fitting and computer plotting skills; taught statistical, measurement analysis and significant number skills; and promoted teamwork.

Particularly in E3, several participants commented on their LSQ fitting and plotting skills. Harry remarked that he was then discussing least squares fitting simultaneously in his honors Chemistry and Mathematics courses, as well as in PHYS 152L. Other participants commented they felt their lab work was improving -- that they were doing better on their written sections in the lab. Joan commented that she had located the sample writing in the manual and that it, along with explanation from the author helped her improve her abstracts considerably. She felt that least squares fitting 'made more sense now,' and that 'she felt more confident, perhaps even a little bored with the apparatus.'

Also in E3, Norma showed particular skill and Žlan with the interface, clicking on various parts and managing to considerably reduce rescaling by planning and mastering some of the more esoteric scaling controls. Finally, during E4, participants claimed to be quite comfortable with least squares fitting procedures. Ian wrote a Lotus 1-2-3 spreadsheet to do all of his fits. Louie claimed to be able to do them 'in [his] sleep'.

Characteristic 2: Curricular Feedback and Pacing Inadequacies

Study participants believe there is inadequate curricular feedback to guide the progress of their learning in PHYS 152L. Student participants and instructors felt there are too few activities, that the three week cycle between their laboratory experiences and graded feedback is too lengthy, and that prelaboratory questions and instructor critiques are unavailable and possibly forgotten when they are required. For example, during E2 student participants forgot basic elements in the control of the software (how to scale, how to print) learned during E1, suggesting that greater access or more frequent access to the software might help.

Also during E2, Harry remarked that he 'wanted the E2 prelab questions [available] during the lab,' and suggested that rather than collect that document from students at the start of the data collection the instructors 'should just check off that it was done' and check a few numbers. Harry also wanted to have the E1 report in hand during E2 data collection (the author returned it to him after data collection for E2 was completed).

Participants during E1 claimed that data collection 'seemed easy,' and felt that the graphical displays were 'good for visualization' (referring back to E1 PLQs). However, when specifically requested to contrast and compare the PLQ predictions, graphs and descriptions to the laboratory data collected, seven of twelve participants were unable to do so -- they simply could not recall their PLQ answers. Two participants specifically requested that they have their PLQs graded and returned before completing the main report.

Characteristic 3: Curricular Content and Lab-Lecture Correlation

Student participants also believe that their laboratory experiences helped illustrate their lecture material. All felt that the lab experiences added considerable depth to their understanding of equations discussed in class. Michelle stated of E4: ''The lab makes you see it better -- it's in depth. In lecture you sleep through it.' and '[lab] looks more real. It gives it more meaning, instead of just the definition.' Louie felt that the rotational motion activity 'showed something from the book where you learned it, but it wasn't really an understanding of the principle.' Ian stated he 'never had labs like these [rotational motion' before, but has seen all of these definitions before.' Two participants even went to the length of claiming that they preferred to learn their physics through initial experience to phenomena in the laboratory followed by lecture, rather than the more typical inverse order.

While all study participants stated that the laboratory activities illustrated their lecture material well and that this was a worthwhile and appropriate role for laboratories, several noted that lecture and laboratory were not always synchronous. For example, student participants who performed E4 during the first scheduled week of the three week lab rotation complained that they had to learn material not yet covered in lecture to complete their PLQs. Those who completed the E4 activity with the author or with the final week of the rotation made no similar claim. This was most pronounced with E4 as E4 covers an enormous amount of lecture material (rotational motion and simple harmonic oscillation) in a very sketchy fashion. As well, the topic of rotational motion is traditionally one of the most difficult topics for student learning in elementary mechanics (Arons, 1990). Unsurprisingly, several participants requested additional laboratory activities on rotational motion.

Participants also perceived that inadequate variety in apparatus (the SONAR - Airtrack System) detracted from learning the different concepts examined in each activity. Participants appeared to have difficulty remaining focused on the changed conceptual content when they used the same apparatus from experiment to experiment. They appeared to relate concepts to concrete apparatus rather than to all of the different physical phenomena (the physical behavior of the apparatus under different constraints). The inclusion of additional experiments employing more varied apparatus would address such problems, and would also provide an opportunity to introduce more kinesthetic and conceptually well-founded activities (Arons, 1990; Laws et. al., 1992). As well, greater opportunity for lab-lecture linkage could be found, and the slow pace between labs could be addressed.

Some curricular topics assumed by the curriculum authors to be trivial background information were not seen as such by students. Most noteworthy was a dearth of student knowledge of introductory statistics (apparantly not part of the standard high school curriculum) and considerable hesitation regarding significant numbers (which is part of most junior high school science curricula). Three participants claimed they had never before seen summation notation, did not recall having ever calculated a variance or standard deviation, and could not interpret the term mean value (all twelve students had previously calculated arithmetic averages). These issues clearly require greater attention.

Characteristic 4: The Commitment to Reflective, Analytic, and Critical Practice

Participants feel that the study itself was a valuable experience for them and that the study itself led to an improvement in their physics learning. This characteristic was confirmed by the author and the instructors. Participant commitment to critical practice and reflective change is characteristic of the reasoning required within the laboratory investigations and (in particular) the measurement analysis curriculum as well as this study. Their participation in this study paralleled the laboratory commitment to the refinement of measurement practices through data collection, observation, reflection and modified practice.

Several student participants took their critical responsibilities for this study as seriously as those required in their laboratory report analyses. Each laboratory report analysis section required students to numerically assess sources of uncertainty in the experimental apparatus and procedure and both qualitatively suggest and quantitatively evaluate the results of using alternative apparatus and techniques. This study demonstrated a similar commitment to rational investigation and managed change in the curriculum.

For example, Louie initiated and carried out an extensive editorial review of the experiment instructions and described how he felt they could be more appropriately reorganized. He also identified less than optimal placement of lab manual figures as leading to student confusion, and later suggested mechanical improvements (removing extra stops and limit guides) to the torsion pendulum apparatus that made it easier to use. These changes were adopted and have become standard practice in PHYS 152L. Other participants reflected upon and described obstacles to their data collection (e.g., the use of an error beep to indicate apparatus calibration completion) and to their learning (e.g., the use of staggered tables when calculating average velocity and acceleration from instantaneous position measurements).

It should also be noted that this situation radically illustrates the incidence of an unusual learning phenomenon felt characteristic of the data collection methodology (by the author). Students were interviewed on their PLQs and lab reports, and answered questions designed to document their practice throughout data collection in the laboratory. All of these activities were in excess of ordinary student experience. The student participants in this study were all spending a great deal more time examining and reflecting upon the PHYS 152L than their peers. The curriculum they experienced was not the equivalent of their peers, and they learned a great deal more as a result.

The results of this commitment to critical practice were also evident to course instructors from descriptions of previous editions of the PHYS 152L curriculum. GTAs participating in the study stated that they felt their experiences with critical analysis and successive refinement in teaching PHYS 152L worthwhile and appropriate. Ideally, we should change the curriculum to better reflect a committment to this kind of reflective activity by making the lab like this study: encouraging critical reflection in all participants and involving all participants (including the GTAs) in critically evaluating their activity in the lab as well as their measurements.

Characteristic 5: Cooperative Learning and Group Interactions

The majority of student participants indicate that they felt working with others on the preparation of joint lab reports made the reporting activity more worthwhile and much more pleasant. Even those participants who initially refused to work with partners (such as Harry) or who had unsatisfactory experiences with a partner (Joan) felt that they had missed out on a valuable experience and indicated they would prefer to work with others in future laboratory activities. Notably, Harry started PHYS 152L indicating that he had detested the thought of working with a partner. After E3, Harry stated that his partner 'learned more' from him than he did from his partner. In the end of course interview, he felt that the author should create a course policy making group reporting mandatory -- 'the course [PHYS 152L] is too passive about working with people.'

Those who had good experiences working with their partners (e.g. Rao) felt that this was amongst the most valuable experiences in the course. Rao said that 'E3 was easier because we did a joint report ...[this was] responsible for [all] the increased ease [performing the data collection and preparing the report].' His partner 'made the lab much more enjoyable,' and he was definitely going to do E4 with the partner and prepare a joint report.

Other students found that group reporting 'degenerated' into purely social conduct and was detrimental to completing the laboratory report writing tasks. Norma and Oscar were highly successful at relating to one another, but mentioned that their interactions were 'too social, indicating they often went off-topic. Norma and Oscar indicated that while they benefitted from checking one another's work, they also encountered difficulty finding free time they could schedule to work together.

Characteristic 6: Conceptual vs. Numeric Analysis of Lab Phenomena

Participants appreciate the mental challenge of conceptual (non numeric) problems as opposed to the more typical alorithmic (plug and chug) problems found in their textbook. This was most particularly illustrated during experiments E2 and E4: E2 had a very open-ended conceptual activity while E4 had almost no conceptual activities. Other activities (e.g., E3) suffered from a lack of linking questions comparing similar theory and measurement. However, these activities can be perceived as too vague and open ended (e.g., by Joan in E2) if the goals are not clearly stated and promoted by instructors. Clearly these kinds of activities need to be better refined and made more pervasive in the curriculum.

Michelle stated (of E2): '[E2 PLQs] made me think about friction -- things that I never really thought of before. [There was] ...more thinking than the E1 PLQs -- the other was just math and graphs.' 'I enjoyed [the] mental challenges.' This perception of E2 was typical amongst the participants. They wanted more opportunity for 'thought' and 'fewer plots'. Student participants particularly enjoyed the last question where they were asked to design investigations examining various possible sources of uncertainty in the measurement apparatus. Oscar commented 'I like designing experiments. In high school we got to design one experiment for credit.' This same student wanted 'bigger questions' of this nature. Even Kevin stated that he thought designing these activities was 'fun' [!]

In contrast, participants described the E4 PLQs as 'just plug-and-chug,' you '...just look up the equation and put the numbers in to the problem,' 'just do what they tell you to,' and (most alarmingly to the author) 'just like in class.' Harry described a sense of finality about E4 -- 'this time in the semester I'm pretty burned out,' 'I didn't have to think so much -- the other prelabs made you think a lot more.' While participants were happy to go through the E4 calculations, they felt something was incomplete in their appreciation of rotational phenomena when they performed labs without conceptual (non-numeric) challenges.

Characteristic 7: General Technical Problems

The critical examination of the curricular activities exposed many shortcomings and technical difficulties. The uncovering and explaining of these problems was turned from an instructor's chore to a task appropriate for each and every person involved in the study by the critical theory/action research paradigm and methodology combination. Participants excelled at the task, uncovering and suggesting remedies and solution approaches for hundreds of individual minor difficulties as described in the preceeding summary.

In the curriculum, confusing directions, faulty procedures and technical glitches abounded. The more striking examples included reversed air tracks, confidence-destroying system beeps, poor manual layout (inappropriate siting of graphics, inappropriate sequencing of instructions for data collection), poor descriptions of reporting procedures and for determination of uncertainties on monitor screen measurements. Participants also spotted mis-labelled axes, disappearing labels, references to missing equipment, poorly worded instructions and questions (e.g., Rao noted that his grade suffered because he had solved all the equations at t=1.0s rather than leaving time as an explicit variable).

Other problems included graphics controls that destroyed all scaling, slow computers and sometimes erratic printers. Many participants suggested highly apporpriate solutions, of which several have since been implemented and are now standard practice in PHYS 152L. For example, in E4, after encountering troubles trying to get the pendulum disk rotated a full 90 degrees, Louie suggested we change the way the safety stops were mounted on the torsion pendulums to allow a much greater rotation angle (while still preventing the disk from moving past 180o for safety). Louie's improvement was subsequently implemented and is now standard practice in PHYS 152L.

Characteristic 8: Shortcomings of Action Research in this Setting

8a. Action research is not an appropriate method for pursuing student alternative conceptions research. During the E3 PLQ interviews, a series of gross difficulties were uncovered in student understandings of mechanical potential energy. Action research did not prove to be a fruitful way of pursing an investigation of the topic, as a thorough theoretical understanding of the taxonomy of alternative conceptions for mechanical potential energy is required before interventionary strategies can be developed.

The last three parts of the first question in the E3 PLQs asked about a situation involving the measurement of potential energy from two different reference points and then asked about the consistency of potential energy measure. The intent was to make clear that the potential energy definition included an arbitrary constant. These question invoked heated debate amongst two participants (partners who were interviewed together) and a great deal of confusion amongst all others. All participants felt they learned more from their questions and discussion with the interviewer than they would otherwise have learned from the question. The topic (potential energy definition) was clearly not appreciated or understood by most participants and was insufficiently addressed in the lecture. The lack of participant responses indicates that this particular subject is worthy of in-depth pursuit through modified Piagetian-style interviews (there is a paucity of research addressing this particular topic).

Student participant interviews with the author all turned into detailed theoretical discussions of what the principal ideas in E3 were supposed to be, and then how they might be approached in the manual. Only one student felt the topic was trivial, and our discussion turned to state functions in physical systems. The remaining discussions resulted in participants listening to me during the interviews rather than the reverse, with little interpretable return data to the study. Participants did not have the basic context for a meaningful examination of the problem, resulting in probes becoming lectures.

8b. The learning of study participants in this study was atypical of the general; PHYS 152L student population. It should also be noted that this situation radically illustrates the incidence of an unusual learning phenomena I feel is characteristic of the data collection methodology. This was previously discussed in the summer pilot reconnaisance work for this study (Chapter 3). Students were interviewed on their PLQs and lab reports, and answered questions designed to document their practice throughout data collection in the laboratory. All of these activities were in excess of ordinary student experience. The student participants in this study were all spending a great deal more time examining and reflecting upon the PHYS 152L than their peers. The curriculum they experienced was not the equivalent of their peers, and they learned a great deal more as a result.

At first, I tried to minimize the impact of the interviews and data collection observations by attempting to keep the language and kinds of inquiries as neutral and random as possible. This helped, but the fact that students had more opportunity to reflect and felt that their role as participants in the study was to reflect meant that their insights into the material were greater than typical. Asking neutral questions during critical moments or situations or time during data collection carried significant import to the participants -- this focused their attention on important data and concepts. Asking questions randomly simply meant there were more occurrences of mental reflection and summary than was typical of student lab practice without the questions.

Hence, it needs to be clearly recognized that student participant experiences in this study should not be claimed generalizable to or representative of typical PHYS 152L students. Their experiences and insights gained were profoundly different. However, characterizing and examining their experiences throughout this study process did result in the construction of many insights and appreciation's of curricular shortcomings and strengths. This knowledge was of great worth informing curricular designers and instructors when reformulating the curriculum and interpreting the experiences of PHYS 152L students in general. As well, the desirable critical elements of their experiences might be made part of the regular curriculum.