SCI 697 Independent Study

Student Conceptual Gain Analysis for PHS 101: Physical Science for Everyday Life

Siobhan Sackey, Northern Arizona University

with Dr. Daniel MacIsaac, SUNY - Buffalo State College

Kathleen D. Eastwood, Northern Arizona University

Kathleen Falconer, SUNY - Buffalo State College

Jim Maxka, Northern Arizona University

Kate Morgan, Northern Arizona University



PURPOSE

  • To compare conceptual gains among three levels of students in three different physics courses, which ranged from greatly reformed to traditional presentation of lecture-based courses.
TOP

NATURE OF THE REFORMS OF PHS 101

  • Developed especially for pre-service elementary teachers
  • An integrated lab and lecture course based on PHS 110 at ASU
  • Curriculum based on AAPT's Powerful Ideas in Physical Science
  • Curriculum aligned to AZ Science Standards, which elementary school teachers will be teaching to
  • Reformed teaching with discourse and interactive engagement (white boarding) PREDOMINANT
  • Real world examples relate to the students' everyday life to develop strong connections with everyday phenomenon.
TOP

NATURE OF OTHER PHYSICS COURSES

  • PHY 111 focuses on classical mechanics, heat, and sound and is the first course in the two-semester, algebra-based, introductory physics sequence. High school physics is a prerequisite for this class and it is intended to be taken with the first semester of algebra. This course FREQUENTLY makes use of seat experiments, white boards and other reformed methods.
  • PHY 161 focuses on classical mechanics and is the first course in the three-semester, calculus-based, introductory physics sequence. High school physics is a prerequisite for this class and it is intended to be taken with the first semester of calculus. This is a traditional, lecture-based course.
TOP

STUDENT and CLASS MAKE-UP

 
 
PHS 101
PHY 111
PHY 161
CURRICULUM Physical Science for Elem Teachers, reform & white board dominant Algebra/Trig-based classical mechanics, some reforms Calculus-based classical mechanics, traditional lecture
CHARACTERISTICS Exreme math anxiety; standard use of coop learning groups Some math anxiety; some use of coop learning groups

Math competent; almost no use of coop learning strategies; passive

GENDER > 75% female roughly 50/50 >75% male
CLASS Fresh / Soph equal among all four years Fresh / Soph

MAJOR

Elementary Education Health Sciences and Biology related Engineering
INSTRUCTOR AVG RTOP SCORE 79 30 20-25*
SIZE 50 - 75 100 + 100 +

*Not all PHY 161 instructors chose not to be formally evaluated with the RTOP instrument.

TOP

EXPERIMENTAL DESIGN

  • Pre-test/post-test design was carried out in the Fall 2000, Spring 2001, Fall 2001, and Spring 2002 in PHS 101, as well as other physics courses. Although the tests varied, there are five mechanics questions that are identical on the PHS 101 test and the tests administered in PHY 111 / PHY 161.
  • All semesters of PHS 101 were taught by teachers who are very familiar with and use reform methods.
  • Fall 2000 and Spring 2001 of PHY 111 were taught by teachers who are very familiar with and use reform methods.
  • Fall 2001 and Spring 2002 of PHY 111 were taught by an instructor just learning reformed methods so Spring 2002 was more reformed than Fall 2001, but not as reformed as in previous years.
  • All semesters of PHY 161 were taught in a traditional, lecture-based manner.
  • The PHS101 test:
    • 30 questions on the test
      • 8 items selected from the Lawson Test of Scientific Reasoning
      • 12 items selected from the Physics Concept Survey (PCS)
      • 6 items from Introductory Astronomy Survey Version 2.0
      • 4 items from the instructor
    • Covering six subject areas
      • Scientific Reasoning - 8 questions
      • Mechanics - 6 questions; 5 FROM THE FCI
      • Electricity and Magnetism - 4 questions
      • Astronomy - 6 questions
      • Optics- 5 questions
      • Thermodynamics - 1 question
TOP

ANALYSIS

  • Gains were calculated by subtracting pre-test scores from post-test scores for each student. The gains were calculated over the test as a whole as well as for each subject area.
  • Normalized gains were calculated by: n = (post - pre) / (#questions - pre)
  • Through the use of Excel and SPSS, correlations were explored between gender and gain as well as final course grade and gain.
TOP

RESULTS

 
 
N
POST-TEST
PRE-TEST
GAIN
GAIN SIG
 
AVG
STD DEV
AVG
STD DEV
AVG
STD DEV
PHS 101
126
3.571 1.148 1.690 1.255 1.881* 1.451 p < 0.001
M
34
3.588 1.351 2.000 1.303 1.588* 1.598 p < 0.001
F
92
3.565 1.073 1.576 1.225 1.989* 1.387 p < 0.001
M-F diff
  0.023   0.424*   -0.401*    
                 
PHY 111
147
2.912 1.140 2.122 1.204 0.789* 1.420 p < 0.001
M
64
3.156 1.042 2.156 1.171 1.000* 1.447 p < 0.001
F
83
2.723 1.182 2.096 1.236 0.627* 1.386 p < 0.001
M-F diff
  0.433*   0.06   0.373    
 
             
PHY 161
236
3.038 1.296 2.538 1.208 0.500* 1.172 p < 0.001
M
163
3.313 1.199 2.755 1.192 0.558* 1.135 p < 0.001
F
73
2.425 1.301 2.055 1.104 0.370 1.260  
M-F diff
  0.888*   0.700*   0.188    

* p < 0.001

Pre-Test Scores

  • In mechanics, pre-test scores are generally
    • lowest for PHS 101 (1.69)
    • higher for PHY 111 (2.12)
    • highest for PHY 161 (2.54)

    This is expected according to typical student preparedness (pre-service elementary teachers vs. college physics vs. university physics).

  • Female student pre-test scores are lower than males scores within each course number, although there is some overlap between PHY 111 males and PHY 161 females.
  • Pre-test gender gaps are
    • least for PHY 111 (0.06)
    • greater for PHS 101 (0.42)
    • greatest in PHY 161 (0.70)

Post-Test Scores

  • In mechanics, post-test scores are generally
    • lowest for PHY 111 (2.91)
    • higher for PHY 161 (3.04)
    • highest for PHS 101 (3.57)

    The difference in post-test scores of PHY 111 and PHY 161 is barely statistically significant

  • Gains and normalized gains, <g>, are
    • least for PHY 161 (<g> = 0.20)
    • greater for PHY 111 (<g> = 0.27)
    • greatest for PHS 101 (<g> = 0.57)

    Normalized gains run exactly counter to pre-test scores by course number.

  • Post-test gender gaps are
    • least for PHS 101 (0.02)
    • greater for PHY 111 (0.43)
    • greatest for PHY 161 (0.89)

    The post-test score gender gap for PHS 101 is statistically insignificant, while the PHY 111 and PHY 161 gender gaps are statistically significant.

  • The gender gap in PHS 101 reduces from a significant pretest gap to an insignificant posttest gap.
  • The gender gaps for both PHY 111 and PHY 161 grew, with PHY 161 doubling the gap present in PHY 111.
  • No correlations were found between gains and final grades.

TOP


CONCLUSIONS

Our data support the following claims:

  • PHS 101 students are least well prepared for conceptually understanding mechanics, followed by PHY 111 students. PHY 161 are best prepared.
  • A gender gap benefitting males exists in the preparation of university physics students for all course numbers in this study.
  • The presence and degree of use of Reformed Teaching methods (i.e. Constructivist inquiry as scored by RTOP typified by extended student discourse incooperative groups via use of white boards) is linked to improved conceptual post-test scores adn strongly linked to pretest-posttest gains.
  • The presence and degree of use of Reformed Teaching methods (i.e. Constructivist inquiry as scored by RTOP typified by extended student discourse incooperative groups via use of white boards) is linked to a reduced or ameliorated gender gap in student's conceptual performance.
TOP

Bibliography

Sackey, S., MacIsaac, D.L., Falconer, K.A. & Eastwood, K.D. (2002). Student Conceptual Gain Analysis for PHS 101: Physical Science for Everyday Life. American Association of Physics Teachers Announcer, 32(2) 129.

Falconer, K & Eastwood, K. " Astronomy Conceptual Gain Analysis for PHS 101: Physical Science for Everyday Life." Talk presented at the summer 2001 AAPT meeting.

MacIsaac, D.L., Cole, R.P. & Cole, D.M. (2002). Standardized testing in physics via the world wide web.
The Electronic Journal of Science Education. 6(3). <http://unr.edu/homepage/jcannon/ejse/ejse.html>

Reformed Teaching Observation Protocol. http://purcell.phy.nau.edu/AZTEC/rtop/RTOP_full/index.htm

MacIsaac, D.L. & Falconer, K.A. (2002). Reform your teaching via the Reform Teaching Observation Protocol (RTOP). The Physics Teacher. 40(8), 479-486.

Powerful Ideas in Physical Science, American Association of Physics Teachers. http://www.psrc-online.org/curriculum/pips/iii.html

Northern Arizona University, PHS 101, PHY 111 and PHY 161 course information: http://www.phy.nau.edu

Special Thank You to the following for their help in this project:

Kathleen Falconer, Kathleen D. Eastwood, Jim Maxka, Kate Morgan, and Nathan Marler.

Funding for this project came from Arizona Teacher Excellence Coalition (AzTEC).