PHY110Y/PHY138Y Laboratory Evaluation
April, 2002

Prepared by David Harrison

INTRODUCTION

Ruxandra and I have recently surveyed students in our PHY110Y/PHY138Y Laboratory about their opinions regarding the lab. This document summarises the results.

549 students participated, which is 60% of the students enrolled in the laboratory.

In the tables below, the numbers in parentheses were the responses to the same question in last year's evaluation. In general, students this year were less positive about the lab than last year. There were two major changes in the laboratory for this year:

1. The "Hoop" experiment was replaced by a series of exercises and readings on Error Analysis.
2. The budget meant that this year's group sizes in the laboratory were significantly larger than at any time in the past.

We believe that the issue of Demonstrator complement for this laboratory has reached a state of crisis, and is probably why the students tended to choose the more negative answers to all the questions in the survey. The students' increased grumpiness is even reflected in their responses to "neutral" questions like:

Do you think the free choice of experiments in the second term is a better educational program than the lock-step that you had in the first term?

The mean on this question increased from 1.82 to 2.01, with larger numbers more negative.

It is interesting to note that the largest increase in negative responses was the overall rating of the Demonstrators: the mean went from 2.50 to 2.93. This is consistent with the large group sizes influencing the results of this evaluation.

MULTIPLE CHOICE QUESTIONS

Question	Mean ± Standard Deviation
Everything considered, give an overall rating of the lab by choosing one of the following: 1 Excellent 2 Good 3 Fair 4 Poor 5 Very bad Comment: below we compare this result to previous years.	3.08 ± 0.97 (2.73 ± 0.86)
Please compare the Physics lab with other labs you are taking: 1 Much better 2 Better 3 The same 4 Worse 5 Much worse Comment: this result is disappointing	3.36 ± 0.98 (3.0 ± 1.1)
Are you learning enough from the lab? 1 Yes a lot 2 Yes, fairly much 3 A little 4 No, not at all	2.80 ± 0.79 (2.52 ± 0.75)
Do you think the free choice of experiments in the second term is a better educational program than the lock-step that you had in the first term? 1 Yes, much better 2 Yes, better 3 They are the same 4 No, it is worse	2.01 ± 0.96 (1.82 ± 0.83)
Does the lab demand too much time from you? 1 Not much time 2 Reasonable 3 A bit heavy 4 Too much time Comment: perfect!	2.43 ± 0.74 (2.39 ± 0.72)
Do you get adequate and prompt feedback on the quality of your work? 1 Yes always 2 For most of the labs 3 Sometimes 4 Never	2.53 ± 0.93 (2.21 ± 0.99)
Do the marks you have received reflect the quality of your work in the lab? In your opinion the marks you have been given are: 1 Too high 2 A bit high 3 Reasonable 4 A bit low 5 Unfairly low Comment: almost perfect!	3.59 ± 0.71 (3.41 ± 0.70)
Please rate your demonstrator in terms of his/her teaching ability and help provided: 1 Excellent 2 Very good 3 Good 4 Fair 5 Poor	2.93 ± 1.27 (2.5 ± 1.2)
At the beginning of the year you read a series of web-documents on Error Analysis, did some computer exercises, and answered some questions on this topic. As an aid to your learning, this material was: 1 Invaluable 2 Useful 3 Not bad 4 Poor 5 Useless	3.04 ± 1.10 (Not asked last year)

Here is a comparison of the overall rating of the laboratory this year and in two previous years:

In the above table, the errors assume that the uncertainty in the number of responses is the square root of the number.

BEST AND WORST EXPERIMENTS

In addition to the multiple choice questions summarised above, we asked the students two related questions:

• Which experiment was the best you have done this year?
• What was the worst experiment you have done this year?

Virtually every experiment in the laboratory appeared as answers to both questions. In all, the laboratory has 46 different experiments: 36 of these were ranked as the "best" and 32 were ranked as the "worst" by at least one student.

We shall summarise the Core experiments first, which many more students have done, and then the Non-Core ones which are done by fewer students.

For the Core experiments, we only report on ones that have 10 or more "votes" as either the best or the worst experiment. The figures in parentheses are from last year's evaluation.

The Score in the final column is:

Score = ( N[best] - N[worst] ) / ( N[best] + N[worst] )

where we assume that the error in the number of "votes" is the square root of the number. Positive numbers are good, and negative ones are bad.

Last year's evaluation indicated dissatisfaction with the Standing Waves experiment, which was substantially revised for this year. Our efforts were clearly under-whelming. We also heavily revised the Air Track experiment, and this work seems to have actually helped the experiment.

For the Non-Core experiments, we only include ones that have 5 or more votes as best or worst.

A FINAL COMMENT

Above we suggested that the very large group sizes in our laboratory influenced the students' responses to this evaluation. A fascinating study has recently been published that shows just how fickle evaluation responses can be. In the study, first year students at Duke were asked to fill out an evaluation of the course both before and after they received their marks in the course. One of the questions asked the students what grade they expected to get.

Not surprisingly, students who expected a higher mark tended to provide more favorable reviews.

However, in the post-course evaluation students who did not receive as good a mark as they had expected significantly lowered their evaluation, while those who received higher marks than they had expected raised theirs. Note that these were the same students rating the same professors in the same courses.

The full article describing the above study, and more, appeared in the New York Times Education Life supplement of April 14, 2002. The article is available on-line but requires a registration with the Times, which is free. The URL of the article is: http://www.nytimes.com/2002/04/14/edlife/14ED-VIEW.html.