Hi Jim,
I tried sending this yesterday, and had a lot of trouble. I never got a copy, so I assume it did not go out.
I tried sending it again this morning and got a message that it was already sent, and I needed to revise it to send it again.
So, here is a revision. If you already received it, please ignore this copy.
Bob
Hi Jim,
As one of the few who voted in favor of the proposal, I want to add my 2 cents to the discussion. I have not had a chance to see read the material you attached, but feel strongly about this issue.
First, and most important, as faculty we must not be opposed to assessment! We insist on assessing our students; those for whom we work should be allowed, in fact encouraged, to assess us. If we are failing in our work, we should have consequences. BUT, assessing us should be based on a clear understanding of what we do.
This starts with a definition of what is the institution's goal. John Carver and his wife Miriam have written two books on not-for-profit governance (the names of which momentarily escape me) in which they focus on three questions: what good, for whom, at what cost! (Peter Drucker has also defined the not-for-profit governance process in similar terms.) The key in their writings is that one must focus on desired outcomes for (in this case) students, not on what faculty do. The first step, then, in assessment is to define the outcomes for the student. This will have the effect of making certain courses more similar than if we all did what we wanted to do. Whether this similarity is in the area of process (critical thinking, writing, etc.) or in the content, or both, is something up for discuussion.
How much these outcomes will differ from institution to institution is debatable as well. Certainly certain content will not differ. Students from all institutions will need to know the answer to 2+2, and they will need to vcome to the same answer. Other content areas may differ according to instutional emphases. Process also should be fairly similar. Problem solving is something all graduates should be capable of - while they may have learned it in different contexts, the basic approaches should not be too different.
Once we agree on outcomes, it is possible to assess whether they are being met. Assessment should be built around the outcomes desired. To the extent these are the same, multi-institutional aassessment is possible; to the extent these differ, it may not be possible. I have no fear about "teaching to the test;" in fact, it is the reverse: we should be testing what is being taught. If faculty are not engaged in fostering the desired outcomes, then testing on them will result in "poor grades" for faculty and/or institutions. Maybe this is a good thing.
As to the impact of tests narrowing the curriculum, this only happens if the testing is the "tail that wags the dog." As curricular changes are made, it becomes necessary to redefine the test instrument. We recently decided that my Law course needed to cover additional materials; when the course changed, my exams also had to change. If assessment need not cover all outcomes, then maybe the tests do not need to change unless outcomes undergo radical changes.
I think the greatest problem in this area is lack of trust. Many faculty do not trust students and vice-versa. Many faculty do not trust administrators and vice-versa. System people do not trust institutional people and vice versa. Why this lack of trust exists and what can be done to change it is a whold different discussion (perhaps more than one).
Second, why we should support (have supported) the proposal presented at the Senate. As I understood Aachim's messages throughout the semester - assessment is coming no matter what we do. The opriginal porposal from the Trustees would have mandated a number of things regardless of our concerns (valid or otherwise). The most recent proposal allowed us some possibility of meaningful input. This latter is better than the former. By rejecting all formns of assessment, we invite the Trustees to categorize us as being "out of control" and this is against our interests (and also those of our students).
Third: some specific thought about your comments:
You make the comment: "One of the most educated persons I know in this community is the least tolerant person I've ever met. He continually writes letters to the editor of our local newspaper saying how tolerance will be the death of our way of life. Would his college say he was successful or not? Would they say his ideas are the desired outcome of their programs?" Perhaps the desired outcome is his ability to engange in exchange of thoughts with the community.
You make the comment: "#5 hits at the real burr I have about assessment. Isn't multi-level assessment what we do now? Isn't grading students assignments and giving them grades in courses just this? Do we need another layer on top of
this? Seems so, but I shake my head in disbelief at the money we will waste and the people's time we will waste to add this extra layer which in the end won't make us any surer of the output than we are now." I think this goes right to the issue of trust. Our students assess whether they have mastered the material. They regulate their study by this means. We independently assess them as a control on their self assessment. Why shouldn't our empoloyers (and funders) exercise the same rights. We all tend to become complacent, sloppy, ... We need outside assessment to make sure we do not lose focus. The money will only be wasted if the additional layer fails to make us any surer; this in turn will occur if the additional layer is not well thought out and/or irrelevant.
Lest this is misinterpreted, let me add that I share the concern that any assessment imposed will be poorly designed. However, I believe this fight needs to be about that issue, and not about whether there will be central and external assessment.
Is this a discussion for a future breakfast group?
Bob
-----Original Message-----
From: Teaching Breakfast List [mailto:[log in to unmask]]On
Behalf Of Greenberg, James ([log in to unmask])
Sent: Monday, April 26, 2004 8:29 AM
To: [log in to unmask]
Subject: Assessment and SUNY
Attached to this posting please find a PDF file by Shavelson on Responding
Responsible to the Frenzy to Assess Learning in Higher Education.
Can you tell me what you think of this?
To get the ball rolling, I tell you what I think (it is quite a ramble...
sorry). I'm no expert in this area, in fact I feel like I shouldn't even
participate in the conversation at times since I know so little... but what
the heck, here goes...
At last weeks Senate meeting the almost unanimous voice of the Senate was
to reject the idea of outside assessment as I understand it. Every argument
that I can remember from this meeting is articulately discussed in this
article. Here are a few of those arguments that I remember and my take on
them.
1- "Assessment often tries to measure filling the bucket, but what about
lighting the candle? Aren't we about both?" Shavelson discusses this
concept with the notion of higher learning vs. domain specific
knowledge. We really need both and are about both. Most people (surveys
suggest) say that what Higher Ed. should be about are the "lighting the
candle" types of things, but assessment only hits at the filling the
bucket. This is one of the issues that is the source of the tension in this
whole assessment argument if you ask me.
Makes sense - most of what I learned in College that I retained as far as
knowledge goes is in Geography. The rest of what I got from College is the
higher level stuff, how to learn, sense of self, awareness of world, etc.
This higher level stuff is what college goal statements are usually about,
but it is the lower level stuff that is usually assessed. The point about
this being a problem with current assessment practice is an excellent one
and that at the very least any assessment we accept should make a clear
distinction between achievements in domain specific knowledge and more
general abilities.
2- "We all know that a standard test, no matter how good, will result in
teaching to the test and narrowing down the curriculum." Shavelson
discusses this issue as well. Standard tests tend to move education
toward a narrowly defined curricula - a one size fits all notion - and away
from the things we perhaps TRULY value in our educational systems. Things
like sense of maturity, ability to get along with people different from
self, etc. A real fear faculty have (and I completely agree with them on
this one) is that standardized assessment will lead to a more narrow
curricula - which in turn we will regret down the road. We will, with our
honest good intentions, undo what we treasure most in our higher
educational system.
3- "I'm highly skeptical that any test or tests, no matter how good, can
really lead to good assessment and a process that betters education."
Shavelson does a nice job with this issue - giving the history of
standardized tests and the thinking behind them. What they are good for
and what they are not good for. For example, do we want to measure only
the relatively permanent knowledge students get, which we know comes from
extensive engagement, practice and feedback like the Pennsylvania Study
tried to do or do we want a more comprehensive assessment that also tries
to measure reasoning, decision making, etc. Or, are we interested in
measuring domain acquired knowledge (why does water make a glacier move?).
These various levels of knowledge are the problem. Some are easier than
others to assess. Some, (the most important ones perhaps) may be nearly
impossible to assess since only a lifetime of experience can bear them
out. Perhaps a survey of alumni over 50 is the best way to assess
these.
In conclusion: His propositions at the end are good, but lack any help.
For example, I completely agree with #1 where we need to assess both
cognitive and social - civic things and we have only been able to focus on
cognitive up to this point. But how do you assess the other? I'm not
pessimistic by nature, but good luck with doing this. Assessing social
things depends on values, culture and morals. If we don't all share the
same ones - and I'm afraid our society doesn't, then what are good and bad
outcomes? An illustration, that might not be necessary to you is,
"tolerance of others opinions." One of the most educated persons I know
in this community is the least tolerant person I've ever met. He
continually writes letters to the editor of our local newspaper saying how
tolerance will be the death of our way of life. Would his college say he
was successful or not? Would they say his ideas are the desired outcome of
their programs?
His #2 is right on the money. We need to hold this conversation -
honestly and openly with the public and trustees and any others in our
society that want to join in. The environment right now seems fairly
toxic to me for such discussions to take place. Why? Why can't we invite
the trustees and others to "town house" discussions about these issues and
also hear what they have to say.
#3 I've already spoken to. We will narrow our curricula and we may not
like what we do to a wonderful system that is working pretty darn well at
the moment. If people are worried about the growing costs of higher
education than let's talk about that. But we shouldn't use assessment to
cut costs, the price is too high.
#4 is right on the money and I would vote for using his conceptual
framework. It is as good as any I've seen. (At least I understand
his!!!)
#5 hits at the real burr I have about assessment. Isn't multi-level
assessment what we do now? Isn't grading students assignments and giving
them grades in courses just this? Do we need another layer on top of
this? Seems so, but I shake my head in disbelief at the money we will
waste and the people's time we will waste to add this extra layer which in
the end won't make us any surer of the output than we are now.
Sorry to ramble and ramble..... but putting this down helped me encode it
better. What do you think? (I bet Achim is listening)
Mr. James B. Greenberg
Director Teaching, Learning and Technology Center
Milne Library
SUNY College at Oneonta
Oneonta, New York 13820
email: [log in to unmask]
phone: 607-436-2701
fax: 607-436-3081
"Ignorance is curable, stupidity lasts forever"
|