Given recent discussions about standardized testing in SUNY, I thought some
on this list might find the following useful/interesting.


Mr. James B. Greenberg
Director Teaching, Learning and Technology Center
Milne Library
SUNY College at Oneonta
Oneonta, New York 13820

email: [log in to unmask]
phone: 607-436-2701
fax:   607-436-3081

"Ignorance is curable, stupidity lasts forever"



            TEACHING TO THE TEST
April 2004
By Lloyd Bond

A recurring criticism of tests used in high-stakes decision making is that
they distort instruction and force teachers to "teach to the test." The
criticism is not without merit. The public pressure on students, teachers,
principals, and school superintendents to raise scores on high-stakes tests
is tremendous, and the temptation to tailor and restrict instruction to only
that which will be tested is almost irresistible.

Although many view teaching to the test as an all or none issue, in practice
it is actually a continuum. At one extreme, some teachers examine the
achievement objectives as described in their state's curriculum and then
design instructional activities around those objectives. This is done
without regard to a particular test. At the other extreme is the unsavory
and simply dishonest practice of drilling students on the actual items that
will appear on the tests.

In addition to offending our moral sense, teaching the actual items on a
test (what James Popham calls "item teaching") is counter-productive for the
very practical reason that it makes valid inferences about student
achievement almost impossible. There is nothing special about the set of
words that happens to appear on a given vocabulary test. We assume that the
words are a sample from a larger population of words, and we want to infer
something about the students' knowledge of this larger set, their general
vocabulary. In like manner, we want to infer that students can solve not
only the particular set of math problems on a test, but that they can solve
an entire class of problems. Drilling students on a specific set of test
items destroys our ability to generalize to this larger domain.

But is teaching to the test all bad? Emphatically not. Consider the coach
who drills young athletes on the very skills they will perform in
competition, or the typing instructor who teaches students precisely the
finger arrangements and keystrokes that will be used in typing. These
practices are not seen as unethical or unsavory for the simple reason that
in these two domains instruction and assessment merge into a single
activity. Indeed, instructing students on anything other than the actual
test itself seems illogical.

The above two examples are so obvious as to be trivial. But more significant
illustrations of the issues are easy to find. In the ambitious New Standards
Project, a national initiative that regularly brought teachers together from
around the country to learn techniques for integrating instruction and
assessment, participating teachers learned to literally merge these two
activities in such a way that they were indistinguishable. Lauren Resnick of
the University of Pittsburgh, one of the visionaries behind the project,
noted that rather than bemoan the inclination to teach to the test, we
should take advantage of it. We should make exercises so compelling, and so
powerful as exemplars of a domain, that honing one's ability to solve them
represents generalizable learning and achievement. Viewed in this light,
teaching to the test is no longer vaguely disreputable because the skills
and knowledge are themselves general and are the very things we wish
students to acquire.

In his senior level psychology course on learning at the University of
Nebraska, professor Dan Bernstein (now at the University of Kansas) was
disappointed in the level of understanding of key concepts that his students
displayed. He decided that the fault might not lie entirely in his students,
but in the way he approached both instruction and assessment. Over the next
few years, he changed his assessment from short abstract essay questions to
problems that asked students to apply concepts in new contexts; added
out-of-class questions about the readings to free up class time for
discussion; and provided web-based examples of responses to test problems,
so that students could learn to identify what makes some answers better than
others. In short, Professor Bernstein merged instruction and assessment in
such a way that "teaching to the test" became an integral part of his craft.
The reader is invited to examine his approach in detail at his online
teaching portfolio.

In its program of advanced teacher certification, The National Board for
Professional Teaching Standards encourages certification candidates to
practice putting together portfolios. They urge candidates to get
suggestions and critical feedback from their colleagues and from others who
have gone through the process. Candidates are encouraged to study excellent
teachers and how they think, write about, and reflect upon their work. The
National Board advises candidates to take several videotapes of their own
teaching, to think about and write critically and reflectively about what
they see. Teachers are encouraged to anticipate the difficulties students
will have with various concepts and how to structure and sequence
instruction to minimize these difficulties. In essence, the National Board
encourages teachers to practice and hone the very things they will be tested
on.

There is a lesson here for teachers and assessment specialists alike. The
tension between the instructional and assessment communities, as well the
pejorative connotations that "teaching to the test" entails, will continue
unabated so long as testing and assessment are seen as something quite apart
from instruction and learning, rather than an integrated reflection of what
was intentionally taught. To paraphrase A. G. Rud of Purdue University, what
is needed is a deliberate attempt on the part of all parties to link
curriculum, instruction, assessment, and standards in a more generative and
even transparent way.