| |
| By Darrell L. Ward, PhD |
| |
| Summary |
| The research results gathered from delivering questions that are aimed towards empowering the learning environment are overwhelming. Yet still, technology tools have neglected the most powerful and universal learning environment, the traditional classroom. This paper will describe the Classroom Performance System (CPS) and its revolutionary support of the traditional classroom as an empowerment tool for the teacher and learner. |
| |
| Unpublished research results by the author (Ward, 1991) in the early 1990s measured the difference between known interactivity in the traditional classroom (30 students - 1 teacher) and that of a technology-empowered classroom (30
students - 1 teacher plus technology). The technology-empowered classroom was over 1000% more interactive than the traditional classroom. These results lay the basis for all efforts towards an interactive technology-empowered learning environment. The learning benefits of questions cannot be achieved if students don't participate in the processing of questions and formulation
of answer(s) to the questions. The CPS- empowered classroom provides the quantitative tools to influence the processing of questions and formulation of answers by the student in a non-threatening and positive manner. |
| |
| Research on the learning benefits of questions has shown that the learning environment can be improved by as much as 150% by the application of various questioning approaches. The use of appropriate questions have proved to (1) engage the learner (2) focus the learner on specific objectives (3) help the learner practice retrieval and application of information to answer questions (4) provide opportunities for feedback allowing students to understand why they missed specific questions (5) provide opportunities for repetition within the learning process. This paper will identify how these techniques can be simply integrated into the CPS empowered classroom. |
| |
| Finally, providing relief to teachers from an administrative perspective
is merely "icing on the cake", with respect to the research results of the
methodology. Teacher turnover and morale have been directly related to increased
administrative duties that take away from what they are trained to do -
teach the students in the classroom. CPS is a proven tool for immediate
and permanent reduction and relief of these administrative burdens. |
| |
| |
| The CPS- Empowered Classroom |
| The CPS classroom is characterized by the following: |
- A one computer classroom - the teacher's computer
- All students are empowered with a simple durable response pad to communicate with the teacher's computer
- A software interface supporting:
- Simple questioning capabilities for involving all students by verbalizing questions
- Feedback mechanism for providing students with absolute knowledge that their answer has been received
- Feedback mechanism denoting class results for each question
- Absolute capture of individual student results
- Delivery and capture of student performance on traditional paper tests
|
| In this environment, each student, regardless of age and background, can easily and without risk of embarrassment, deliver their answers to objective questions. These questions can be: |
- Constructed by the teacher "on the spot" and verbalized to the class
- Written on the board by the teacher
- Referenced by the teacher from any printed materials
- Constructed by teams of students and presented to other teams of students in a collaborative mode
- Pre-developed by the teacher and projected to the class from CPS
|
| A large television monitor and/or a data projector connected to the teacher computer can support the environment visually. |
| |
| There are currently around 7000 such CPS-empowered classrooms. CPS classrooms reside in all 50 states within the United States and several foreign entities, including the United Kingdom, Puerto Rico, Canada, Australia, Singapore and the Netherlands. The CPS-empowered classrooms range from kndergarten classrooms in the K-12 environment to graduate classrooms in the higher education environment. |
| |
| |
| The Foundation of an Interactive Learning Environment - Student Engagement |
| Clearly, no learning will take place if students are not engaged and motivated to learn. Research by William Sanders (Sanders, 1999) in support of the Tennessee Department of Education has identified the most important aspect of academic progress is the teacher. CPS is a major teacher tool with the following important characteristics to support the teacher: |
- CPS provides a tool for the teacher to positively and simply engage all students in the class at any point in the instructional process
- CPS provides both immediate and delayed feedback to the teacher and/or the students in the class
- CPS reduces the cost to the teacher of analyzing short-term and long-term results of classroom interactivity
- CPS supports the repetition of core learning concepts
|
| Certainly, if Sander's research is accepted, any tools that provide teachers
more time to devote to student academic progress are extremely important.
CPS addresses this by supporting the research documented to improve the
learning environment as well as reducing the teachers' relatively non-productive
time accommodating the various administrative activities. |
| |
| With the key component in academic progress being the teacher, arming
the teacher with CPS helps to make student engagement during the instructional
process become the objective. CPS achieves that by providing each student
with a simple and wireless handheld, the CPS response pad. This pad is non-threatening,
robust, and is in use (at the time of this white paper, more than 250,000
CPS response pads are in this market) from pre-K through graduate education. |
| |
| The engagement of the student on a continuous and no-risk basis clearly
changes the classroom paradigm. CPS supports the teacher with a simple interactive
mechanism for achieving classroom interactivity for all students in the
classroom. This classroom environment assumes a teacher computer, CPS software,
wireless handhelds for each student, and a recommended but optional projection
system consisting of a video projector or TV. This mechanism, in its simplest
form, is as follows: |
- The teacher delivers an appropriate question to the class by verbalizing
the question, projecting it on an overhead, writing it on the chalkboard
or flipchart, or projecting it via computer software such as PowerPoint
or CPS.
- The teacher provides objective answers from which the students may
choose their answer.
- Students answer with their handheld CPS pads with affirmation on the
computer projected screen or with verbal affirmation by the teacher
if no projection is available.
- The results are typically provided to the teacher with the number
of responses for each answer option shown next to the option or with
a histogram. Normally, CPS will deliver the percentage correct for the
question along with the cumulative percentage correct for the session.
|
| Since students have no risk of embarrassment with respect to their individual
answers, they are very motivated to participate. Many CPS-empowered teachers
will provide ownership to the students by working with them to set a cumulative
scoring goal for the class and then tracking their class progress in relation
to that goal. If the class has an achievable learning session goal and incentive
to reach that goal, student ownership of learning can be provided to every
group learning session. |
| |
| Results by Ryan Delaney (2002) in Ohio, where student ownership of learning
has been an integral part of the process, have been very impressive in the
K-5 environment. The student academic progress of several schools employing
the concept of goal setting and data collection based on student engagement
is shown in Appendix A. |
| |
| |
|
Learning Benefits of Questions and CPS |
| Research over a long period of time has documented the best practices
and benefits of questions as a learning tool. Of course, the assessment
tool for utilizing questions and answers to evaluate student knowledge has
been prevalent for centuries. This paper will summarize the learning benefits
of this research while coordinating the basic research results to corresponding
CPS tools. |
| |
| |
| Engaging the Students |
| Thalheimer (2003) has summarized almost a century of research on the learning
benefits of questions. First and foremost, any question is of no benefit
to the learners and teacher unless the learner processes the question and
participates in the learning activity. Learners who are motivated to answer
questions are more likely to learn than those that are not motivated (Frase,
1971; Frase, Patrick & Schumer, 1970). CPS provides motivation for every
student to answer individually while also providing a group incentive for
processing and answering questions. |
| |
|
Individuals are motivated to answer as CPS captures a complete record
of their performance. Additionally, peer pressure is applied in a subtle
manner, as a large majority of the class will typically answer every question.
|
| |
|
Group incentives can lead to improved individual motivation because the
learners will want to contribute to the overall success of the group. With
a quantitative and real-time feedback tool present in the classroom, student
motivation to participate is heightened. The teacher can announce "if our
group cumulative score for this learning session is above 80, we will have
no homework tonight" providing individual and group incentives to participate.
Since the cumulative data is continuously available, everyone can strive
to reach or stay above the group goal with all learners actively participating. |
| |
| |
| Feedback, Interactivity and Learning |
|
With CPS, feedback is available at every learning opportunity. Clearly,
feedback seems intuitive as a factor to improving the learning environment
and all teachers' strive to create an interactive environment. In fact,
research validates the importance of feedback in producing learning benefits
(Bangert-Drowns, Kulik, Kulik, and Morgan, 1991). |
| |
|
The work of Mazur (1995, 1996), with his Peer Review approach in Physics,
attempts to deliver questions to the class that will create a 50% correct
response (Mazur, 1998). Thus, corrective feedback is provided to approximately
half of the class. Guthrie (1971) claimed huge learning improvements when
it is provided for incorrect answers or in a corrective manner. |
| |
| |
|
Question Repetition |
|
Boyd (1973) showed that multiple questions on a single learning point
improved learning results over single questions by 29%. CPS can assist on
the evaluation of the need for question repetition since the results become
available in real time. |
| |
|
Clearly, when the results of a question are overwhelming positive, one
can decide that an additional question may not be necessary. On the other
hand, if the results are "marginal" from the teacher's perspective, then,
after corrective feedback, an additional question is most likely in order.
|
| |
| |
|
Learning Benefits |
|
Thalheimer (2003) has quantified the learning improvements as follows: |
| |
| Learning Factor |
Representative Minimum Improvement |
Representative Maximum Improvement |
|
Retrieval PracticeRetrieval practice-even without feedback- has
been found to improve learning results. |
30% |
100% |
|
FeedbackFeedback produces better learning results than no feedback.1 |
15% |
50% |
|
Focus AttentionPrequestions and learning objectives produce similar
effects and improve learning outcomes when presented before learning
materials.2 |
5% |
45% |
|
RepetitionWhen meaningful repetitions of learning materials are
provided, learning results are better than if no repetitions or fewer
repetitions are provided.3,4 |
30% |
110% |
| Total Improvements |
40% (a) |
150% (b) |
|
|
a, b = to be conservative, the column totals are divided by 2 |
|
- Delayed feedback produces better learning than immediate feedback by 10 to 25%
- These numbers represent research on both non-inserted prequestions and learning objectives.
- When initial repetitions are compared to no repetitions, the representative minimum is 30%, and the maximum is 110%. When subsequent repetitions are compared to fewer repetitions, the representative minimum is 15%, and the maximum is 40%.
- Although questions inherently utilize some spacing, this effect may be small because one repetition with minimal spacing may not produce a large result, and certainly not the 5 to 40% improvements typically created by spacing. To be conservative, spacing improvements are not used in the overall calculation
|
| eInstruction's Methodology for Interactive Data Driven Instruction |
| With the basic results provided above, eInstruction has developed an instructional approach or methodology that attempts to maximize the learning benefits of questions. This methodology is encapsulated in eInstruction's Interactive Data Driven Instruction model (Ward, 2003) and supported by the CPS empowered classroom. |
| |
| |
| Daily Instructional Activity |
|
We recommend, as a general guideline, that 8-12 appropriate objective
questions be integrated into every one hour learning activity. The break
down of the questions, assuming the maximum of 12 questions is as follows: |
- Prequestions (4)
- 2 review questions over previous materials to provide retrieval and repetition practice
- 2 prequestions to direct attention to the session's learning objectives
- Inserted questions (5)
- Questions for retrieval and repetition purposes
- Postquestions (3)
- Questions for retrieval and repetition practice (spacing occurs which improves learning benefits)
|
|
With CPS as an active supporting mechanism, real-time instructional decisions
can be implemented based on the data that is collected and provided during
this process. Since the data is collected in real-time, this data can drive
or direct the instruction in real-time to support repetition, retrieval,
and spacing decisions for additional questions that may be used. |
| |
| |
|
Bi-weekly Instruction Activity |
|
On a bi-weekly basis, it is recommended you incorporate a fun team activity
into the learning environment. eInstruction has developed a CPS team activity
called "There It Is!" to support a simple and fun retrieval/repetition activity.
Basically, each team has a set of objective questions in hard copy format
numbered from 1 to some reasonable number, let's say 30. Each question has
a set value and a bonus value. Prior to the game, the teacher will input
a key for the game providing the correct answer for each question. |
| |
|
Each round of the game is initiated by the teacher. CPS will randomly
select a question number from the 30 questions and assign that question
to a team. Each team will immediately identify that numbered question from
their hardcopy of the questions, process that question and respond with
their answer. When all the teams have provided an answer, the teacher will
end the round and the scores will be shown. |
| |
|
Of course, teams that missed the question will not receive any points.
The teams that get the correct answer will receive the point value for correct
answers. The first team to get the correct answer to their question, on
their first attempt, will receive the bonus points. |
| |
|
The game proceeds on that basis until a predetermined number of rounds
have been completed. If there is a tie, then a sudden death can proceed
to determine a winning team. |
| |
|
This activity integrates fun, competition, teamwork, repetition, retrieval,
spacing and delayed feedback into the learning environment. The questions
would be focused on the learning points of the previous 2-3 days. Additionally,
delayed feedback can be accomplished by revisiting questions, if any, which
were missed by any team. |
| |
| |
|
Weekly Instructional Activity |
|
On a weekly basis, the methodology identifies a self-paced classroom activity
with immediate feedback. This is diagnostic in nature with practice on retrieval,
repetition and spacing. Again, CPS supports this by coordinating a teacher-entered
key to a set of objective questions in hardcopy format. It is recommended
that a relatively small number of questions be used and coordinated to the
key learning points of the week. Approximately 10 - 15 questions are reasonable
to support this activity. |
| |
|
Each student proceeds to answer the questions. As an answer to a question
is entered the student is provided feedback in the following manner. If
the answer to the question the student is attempting is correct, CPS will
blink the student's response pad number and then move immediately to the
next question. The feedback is positive - the answer entered was the correct
answer. |
| |
|
On the other hand, if the answer provided is incorrect, CPS will blink
the student's response pad number, but remain on the question. The feedback
indicated that the answer entered was incorrect and another answer must
be entered until the correct answer is provided. At that time, the student
is positioned on the next question and ready to enter an answer for that
question. |
| |
|
Although the feedback is very minimal, correct or incorrect, the student
can eventually attempt every question and identify the correct answer to
every question. This can provide the basis for additional work for the student
to understand the basis for the correctness of a question. |
| |
| |
|
Bi-Monthly Instructional Activity |
|
Twice a month, we suggest a more formal 30-50-question test. This test
can be constructed in the same manner as the Bi-weekly activity with a hardcopy
for the student and a CPS teacher-constructed key. The key benefits of spacing,
repetition, retrieval and delayed feedback are applicable with this activity. |
| |
|
Notice the delayed feedback approach recommended here. As the student
undertakes the questions, no feedback is provided other than the fact that
CPS has received the student's answer. Since the results are electronically
captured there should be no reason why the feedback for the activity cannot
be provided within a day or two of the activity. |
| |
|
Many times activities like these require so much teacher work that the
results of the activity are not available for several days or perhaps a
week. Research shows that delayed feedback beyond a "reasonable" time of
a day or so provides no learning benefit. |
| |
| |
|
Teacher Productivity in the CPS-Empowered Classroom |
|
Clearly, the more data that is collected from instruction and assessment
activities, the more time and effort is required to the grading and evaluation
of student performance. Teachers must carefully balance the capture and
evaluation resources against the actual time delivering instruction and
imparting knowledge. A technology that produces an abundance of quality
student data from the instructional and assessment process while delivering
more time for teachers to focus on actual instruction is a compelling technology.
CPS does precisely that as it seamlessly assists the instructional process
while reducing teacher time spent on the mundane grading and analysis activities. |
| |
|
The methodology described above delivers a tremendous return on investment
with respect to the time of teachers. Since all aspects of the methodology
outlined above are based on existing teacher objective materials, the efforts
to coordinate CPS answer keys to these materials are very minimal. |
| |
| |
|
The returns on those minimal efforts are significant and include the following: |
- Complete reporting on student performance
- Standards-based reporting
- Immediate and delayed feedback to students and teacher
- Repetition on key learning points
- Retrieval of information in a variety of scenarios
- Spacing of questions to maximize learning
- Grading, grade entry and grade reporting electronically
- Availability of student performance data to district administrators seamlessly
|
| Summary |
|
The empirical data supporting the use of questions as a learning tool
is overwhelming. With that as a basis, this white paper demonstrates, from
both a productivity and instructional perspective, how CPS supports and
extends the learning environment. The foundation of the learning benefits
of questions is the actual engagement of the students. This paper demonstrates
the value and a methodology for continuous engagement of students in the
learning environment. |
| |
|
An eInstruction methodology with CPS support for each application is presented
which consists of: |
- Daily instructional usage
- Bi-weekly collaborative usage
- Weekly student practice with immediate feedback
- Bi-monthly student assessment with delayed feedback
|
|
Each application of CPS, within the methodology, is supported by extensive
data capturing and analysis capabilities. These features provide tremendous
instructional support for the classroom teacher while reducing their mundane
administrative duties. |
| |
| |
|
References |
|
Bangert-Downs, R. L., Kulik, C-L. C., Kulik, J.A., and Morgan, M. (1991).
The instructional effect of feedback in test-like events, Review of Educational
Research, 61, 213-238. |
| |
|
Boyd, W.M. (1973). Repeating questions in prose learning. Journal of Educational
Psychology, 64, 31-38. |
| |
|
Delaney, Ryan. (2002). Unpublished study. |
| |
|
Frase, L.T., Patrick, E., and Schumer, H. (1970). Effect of question position
and frequency upon learning from text under different levels of incentive.
Journal of Educational Psychology, 61, 52-56. |
| |
|
Guthrie, J.T. (1971). Feedback and sentence learning. Journal of Verbal
Learning and Verbal Behavior, 10, 23-28. |
| |
|
Hake, Richard R. (1998). Interactive-Engagement vs Traditional Methods:
A Six-Thousand-Student Survey of Mechanics Test Data for Introductry Physics
Courses. American Journal of Physics, Vol 66, pps 64-74. |
| |
|
Irby, T. and Ward, D. (1981). An educational environment model based on
a micro implementation of Hypertext. Proceedings of ADCIS, March 1981. |
| |
|
Massen, C., Poulis, J., Robens, E., Gilbert, M. (1998). Physics lecturing
with audience paced feedback. American Journal of Physics, Vol 66, May 1998. |
| |
|
Mazur, E. (1995). Newton, One-on-One. Harvard Journal. |
| |
|
Mazur, E. (1996). Peer Instruction: A Users Manual. Prentice Hall, 1996. |
| |
|
Sanders, William F. (1999). Teachers, Teachers, Teachers. Blueprint Magazine. |
| |
|
Thalheimer, W. (2003). The Learning Benefits of Questions. A Work Learning
Research Publication (www.work-learning.com). |
| |
|
Ward, D. (1977). A Computerized Lecture Preparation and Delivery System.
Journal of Educational Technology, Vol. 6(1), 1977-1978. |
| |
|
Ward, D. and Bush, S. (1980). Hypertext-a generalized educational computer
tool. Proceedings of the National Educational Computer Conference, pp. 19-24,
1980. |
| |
|
Ward, D. (1991). A Quantitative Analysis of Interaction in the Classroom.
Unpublished study. |
| |
|
Ward. D. (2003). Interactive Data Driven Instruction. A Professional Development
Seminar. |
| |
|
Appendix A |
| |
|
Results from Ryan Delaney Application of Student Engagement and Accountability
Approach |
| |
 |
| |
 |
| |
 |
| |
 |
| |
 |
| |
 |
| |
 |
| |
