| |
| J. Poulis and C.
Massen |
| Department of Physics,
Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The
Netherlands |
| |
| E. Robens |
| Department of Chemistry,
Gutenberg University, Mainz, Germany |
| |
| M. Gilbert |
| Department of Physics,
University College London, Gower Street, London WCIE 6BT, United Kingdom
(Received 18 November 1996; accepted 29 August 1997) |
| |
| For some time we have
experimented at the Eindhoven University of Technology with a system which
provides students with an electronic feedback path to the lecturer, audience
paced feedback (APF). In this paper we describe this APF system, and give
indications of its effectiveness. 1998 American Association of Physics Teachers.
|
| |
| |
| I. INTRODUCTION |
| There is a considerable
body of evidence supporting the view that traditional lecturing, where the
students receive the lecturer's wisdom in full flow with only an occasional
lapse into interaction, is flawed. Attempts have been made in recent years
to use technology in the form of computers and multimedia to improve upon
the lecture, but, while clearly of growing importance, these are also encountering
constraints. In this paper we describe the use of a different technique
to improve lectures. |
| |
| |
| II. OUTLINE OF THE
SYSTEM |
| Audience paced feedback
(APF) is the provision, for each student in the lecture theatre, of an electronic
handset to allow him/her to answer simple binary questions from the lecturer. |
| |
| |
| A. Handsets |
| On each handset is
a single button, which is only pressed to indicate a "yes" response to an
inquiry from the lecturer. This is for reasons of operational simplicity:
it requires little effort to understand the system, there is no possibility
of the wrong button being pressed, and it causes minimal disruption and
distraction when used. The students' opinions are then transmitted to the
central receiver along hardwired connections. |
| |
| |
| B. Reception and
display |
| The responses are then
collated and displayed as a simple percentage readout, visible to all participants
in the lecture. The system is calibrated using a button on the central receiver;
the lecturer asks all present to press their buttons and this number of
replies is filed as 100%. In addition, the central unit also has a button
to zero the display after each set of responses. |
| |
| |
| III. FACETS OF AN
APF LECTURE |
| The system introduces
a more active role for the student in lectures, by giving them a simple
feedback path to the lecturer, which should increase their cognitive engagement.
This method of interaction is fundamentally different from the traditional
"raise your hand to answer" style. The crucial differences are that: |
| |
| All students are involved
in answering any given question. |
| |
| The lecturer can ask
multiple choice questions allowing the formative nature of misconceptions
to be explored - the students' replies are anonymous (so they need have
little fear of being identified and ridiculed by their peers). |
| |
| The procedure moves
the format of the lecture away from the chalk and talk model and closer
to that of a seminar. |
| |
| |
| A. Question format |
| Questions are asked
at frequent intervals and the nature of the inquiry allows the polling of
negative replies. The purpose of questions include: |
| |
| (a) exploration,
to gauge the opinion of the students of their own understanding (for example,
"Have you understood my arguments regarding this equation?"); |
| |
| (b) verification,
to allow the lecturer to assess the state of the students' comprehension
("Does this apply to high temperatures?"); |
| |
| (c) interrogation,
to test the ability of students to apply the work to specific situations
(for example, a multiple choice question for which the student has to decide
on the correct equation to use and hence find a solution; where each answer
is presented in turn and the students press the button when their solution
is presented |
| |
| (d) organization
("Are you ready for me to continue?"). |
| |
| When a type c problem
is presented the lecturer pauses and asks the students to press their buttons
when they feel they have an answer (that is, he/she asks an interim type
d question) and waits until 60-70% of the students have indicated that they
do. This figure is a product of experience, and balances the need to give
students time to answer and the boredom of those who have already completed
the task. During this period students are free to discuss the problem among
themselves. |
| |
| When enough are ready,
the lecturer then presents each of the possible answers in turn and the
proportion of those answering "yes" to each is seen (the normal method for
answering questions of type c). |
| |
| |
|
Examination pass rate, %, and sample size, S
|
| |
| |
Year
A |
Year
B |
Year
C |
Year
D |
Year
E |
Year
F |
| Subject |
%.......S |
%.......S |
%.......S |
%.......S |
%.......S |
%.......S |
|
Maxwell
Theory
|
38..307 |
47..259 |
83..269 |
87..230 |
..... |
..... |
Vibration
and Waves |
..... |
..... |
87..106 |
87..122 |
93..100 |
84..115 |
Optics
and
Maxwell Theory |
..... |
..... |
76..165 |
65..180 |
56..134 |
57..190 |
Mechanics
and
kinetic theory
of gasses |
42..72 |
60..84 |
93..116 |
96..112 |
..... |
..... |
Kirchhoff,
Mechanics and
Vibrations |
34..148 |
39..124 |
95..107 |
84..98 |
..... |
..... |
Engery
Management |
..... |
..... |
82..270 |
74..265 |
68..85 |
61..255 |
|
Statistical
Thermodynamic
|
..... |
..... |
98..249 |
88..261 |
53..390 |
54..578 |
|
| Table I. The percentage
pass rate and sample size around a period where APF was used. Years C and
D are those in which APF has been used, shown in bold; years A, B, E, and
F are the results of traditional lectures. |
| |
| |
| B. Lecture format |
| At the beginning of
each lecture course the students are given a short talk on how the system
works, to ensure a minimal level of confusion in its subsequent operation. |
| |
| At the start of a lecture,
the students are given photocopies of the overhead projections (OHPs) to
be used. These OHPs use a basic color coding scheme: green to signify that
the material is being repeated or is expected prerequisite knowledge, red
to show questions, blue for the possible answers, and black for new information.
The lecture itself begins, as many do, with a recapitulation of what was
said during the previous one. Questions are asked of the students to test
the understanding and recall of the matter, with response via APF. |
| |
| Throughout the lecture,
a difficulty in comprehension is signaled by there being more than 30% incorrect
answers to a question of type a, b, or c; or if the time taken for students
to signify their readiness is too long for a question of type d (the average
wait is around 1 1/2 min, but this, of course, varies with the complexity
of the question being posed). When this occurs, the lecturer takes the students
through the problem step by step and then asks a supplementary question
(often on the blackboard) to check the new level of student understanding. |
| |
| If it becomes obvious
that the students are completely comfortable with a topic, for example,
by questions being answered very rapidly and correctly, the redundant questions
are discarded. |
| |
| At the end of a lecture,
the teacher asks "Who thought that was too fast/too slow/about right?" A
desired distribution would be approximately 20% "too fast," 20% "too slow,"
and 60% "about right," as it is impossible to match everyone's ideal speed
of presentation. |
| |
| Overall, a lecture
consists of around 20 min of APF functions interspersed between 25 min of
conventional lecturing. |
| |
| |
| IV. RESULTS |
| It is difficult to
measure the interest level of students in a lecture empirically, but there
is some evidence that they prefer the use of APF. In addition, there are
data showing that the use of the system improves examination results: while
it is generally accepted that examinations are not a perfect measure of
student comprehension, they are a reasonable indicator of understanding. |
| |
| These results are not
immune to any Hawthorne effect with respect to the students-they enter a
new exciting lecture format and there may be bias from this. However, the
system has been in use in Eindhoven for a considerable time (since 1966),
so the lecturers using it have become very familiar with the system, mitigating
any effects on them. |
| |
| |
| A. Opinions |
| The 288 APF-exposed
and 19 790 "ordinary" students were asked on a Likert scale from one to
nine (nine indicating a very strong positive): "Do lectures contribute much/
little to a better understanding of the subject?" The mean score for non-APF
students was 5.1; for APF students this rose to 6.7. This indicates a preference
for the lecture when APF is present, and a positive reaction to it. |
| |
| |
| B. Pass rate |
| The end-of-course examination
pass rate for a given course was measured over four years: either two non-APF
years followed by two with APF; or two with feedback and the two subsequent
traditional years. In each case the academic year of each of the four years'
students remained the same. The data are drawn from the period 1979 to 1992,
and cover the faculties of industrial engineering and management science,
electrical engineering, chemical engineering and chemistry, and applied
physics. |
| |
| To ensure consistency
in the standard of understanding represented by an examination pass in APF
courses during the periods considered, an independent supervisor from each
faculty was appointed. It was felt vital that the course present the same
amount of material, the supervisor be closely consulted when examinations
were designed to ensure their consistency, and he/she be ultimately responsible
for the courses' year-to-year equivalence rate. |
| |
| Table I shows that
the use of APF improves the pass in most of the variety of physical science
lectures in which it has been used. The mean pass rate, Fig. 1, of the APF
lectures is significantly higher than that where conventional methods have
been employed. Of equal importance is the reduction in the standard deviation
of this average, indicating a more consistent level of comprehension throughout
any given class, and year on year. This in turn means that decisions on
the required level of understanding assumed for future courses can be made
with more confidence. |
| |
| |
| V. CONCLUSION AND
DISCUSSION |
| These results demonstrate
that the application of APF in the lecture theater has been of significant
use in the students' learning process; both increasing the mean pass rate
of individuals exposed to it and reducing the variability between the achievements
of different students. |
| |
| These are being tentatively
ascribed to four effects, which are, in order of decreasing importance:
the removal of the "house of cards" effect, the negation of the inherent
passivity of students in lectures, student-student teaching, and a mild
Hawthorne-like effect. |
| |
| In a traditional lecture
it can be extremely difficult to measure the students' comprehension of
a topic. This can lead to a house of cards effect, where the lecturer is
explaining a subject to students who have yet to understand its precursor.
APF allows the lecturer to ensure that the majority of the student body
has understood the material before moving on. |
| |
| In addition to this,
the students are given a role in the lecture, and play an active part in
it. This increases their cognitive engagement and so material taught to
them is considered more closely. |
| |
| The students also spend
some time discussing each problem; during this period there is some element
of student-student teaching, or consolidation of material presented by the
lecturer between students. |
| |
| Finally, there is likely
to be a residual Hawthorne-like effect: the students are presented with
a situation in which the lecturer has prepared a clear set of OHPs, and
where they are given the special attention of the handsets. This cannot
be completely discounted as an explanation until far larger longitudinal
studies are undertaken. |
| |
| With the results of
other independent studies involving student feedback also showing promise,
further exploration of the efficacy of lecture feedback will be conducted,
with the aim of improving this teaching form. |
| |
| |
| ACKNOWLEDGMENT |
| The
authors are grateful to Professor Sir Eric Ash for a number of helpful discussions.
|
| |
| Corresponding author. |
| |
| For example, 1. A.
Halloun and D. Hestenes, "The initial knowledge state of college physics
students," Am. J. Phys. 53, 1043-1055 (1985); M. Jackson and M. T. Prosser,
"Less Lecturing, More Learning," Stud. Higher Educ. 14 (1), 55 (1989); F.
Costin, "Lecturing versus other methods of teaching: A Review of Research",
Br. J. Educ. Technol. 3 (1), 4-31 (1972). Many are identified in Diana Laurillard,
Rethinking University Teaching: A Frameworkfor the Effective Use of Educational
Technology (Routledge, London, UK, 1993), pp. 120-178. |
| |
| For example, "Is this
equation dimensionally incorrect?" At a trivial level, simply looking to
see which students are facing the lecturer only measures that item, and
not whether the student is listening. |
| |
| Studied in F. J. Roethlisberger
and W. J. Dickson, Management and the Worker (Harvard U.P., Cambridge, MA,
1946), pp. 3-604; further discussed in H. M. Parsons "What happened at Hawthorne?"
Science 183, 922-932 (1974); and G. Adair, "The Hawthorne effect: A reconsideration
of the methodological artifact," J. Appl. Psych. 69 (2), 334-345 (1984).
These data are drawn from the Teaching Evaluation Scheme at TUE. |
| |
| The standard deviations
of these are not recorded. Before 1979 the data are no longer available,
and after 1992 there are no relevant lecture variations. These data are
independent of the data provided by Ref 6 (which was from a separate study
regarding the entire university, hence the disparity in sample size). For
example, E. Mazur, Peer instruction A Users Manual (Prentice-Hall, Upper
Saddle River, NJ, 1997), pp. 3-42. |
