Attached to this posting please find a PDF file by Shavelson on Responding Responsible to the Frenzy to Assess Learning in Higher Education. Can you tell me what you think of this? To get the ball rolling, I tell you what I think (it is quite a ramble... sorry). I'm no expert in this area, in fact I feel like I shouldn't even participate in the conversation at times since I know so little... but what the heck, here goes... At last weeks Senate meeting the almost unanimous voice of the Senate was to reject the idea of outside assessment as I understand it. Every argument that I can remember from this meeting is articulately discussed in this article. Here are a few of those arguments that I remember and my take on them. 1- "Assessment often tries to measure filling the bucket, but what about lighting the candle? Aren't we about both?" Shavelson discusses this concept with the notion of higher learning vs. domain specific knowledge. We really need both and are about both. Most people (surveys suggest) say that what Higher Ed. should be about are the "lighting the candle" types of things, but assessment only hits at the filling the bucket. This is one of the issues that is the source of the tension in this whole assessment argument if you ask me. Makes sense - most of what I learned in College that I retained as far as knowledge goes is in Geography. The rest of what I got from College is the higher level stuff, how to learn, sense of self, awareness of world, etc. This higher level stuff is what college goal statements are usually about, but it is the lower level stuff that is usually assessed. The point about this being a problem with current assessment practice is an excellent one and that at the very least any assessment we accept should make a clear distinction between achievements in domain specific knowledge and more general abilities. 2- "We all know that a standard test, no matter how good, will result in teaching to the test and narrowing down the curriculum." Shavelson discusses this issue as well. Standard tests tend to move education toward a narrowly defined curricula - a one size fits all notion - and away from the things we perhaps TRULY value in our educational systems. Things like sense of maturity, ability to get along with people different from self, etc. A real fear faculty have (and I completely agree with them on this one) is that standardized assessment will lead to a more narrow curricula - which in turn we will regret down the road. We will, with our honest good intentions, undo what we treasure most in our higher educational system. 3- "I'm highly skeptical that any test or tests, no matter how good, can really lead to good assessment and a process that betters education." Shavelson does a nice job with this issue - giving the history of standardized tests and the thinking behind them. What they are good for and what they are not good for. For example, do we want to measure only the relatively permanent knowledge students get, which we know comes from extensive engagement, practice and feedback like the Pennsylvania Study tried to do or do we want a more comprehensive assessment that also tries to measure reasoning, decision making, etc. Or, are we interested in measuring domain acquired knowledge (why does water make a glacier move?). These various levels of knowledge are the problem. Some are easier than others to assess. Some, (the most important ones perhaps) may be nearly impossible to assess since only a lifetime of experience can bear them out. Perhaps a survey of alumni over 50 is the best way to assess these. In conclusion: His propositions at the end are good, but lack any help. For example, I completely agree with #1 where we need to assess both cognitive and social - civic things and we have only been able to focus on cognitive up to this point. But how do you assess the other? I'm not pessimistic by nature, but good luck with doing this. Assessing social things depends on values, culture and morals. If we don't all share the same ones - and I'm afraid our society doesn't, then what are good and bad outcomes? An illustration, that might not be necessary to you is, "tolerance of others opinions." One of the most educated persons I know in this community is the least tolerant person I've ever met. He continually writes letters to the editor of our local newspaper saying how tolerance will be the death of our way of life. Would his college say he was successful or not? Would they say his ideas are the desired outcome of their programs? His #2 is right on the money. We need to hold this conversation - honestly and openly with the public and trustees and any others in our society that want to join in. The environment right now seems fairly toxic to me for such discussions to take place. Why? Why can't we invite the trustees and others to "town house" discussions about these issues and also hear what they have to say. #3 I've already spoken to. We will narrow our curricula and we may not like what we do to a wonderful system that is working pretty darn well at the moment. If people are worried about the growing costs of higher education than let's talk about that. But we shouldn't use assessment to cut costs, the price is too high. #4 is right on the money and I would vote for using his conceptual framework. It is as good as any I've seen. (At least I understand his!!!) #5 hits at the real burr I have about assessment. Isn't multi-level assessment what we do now? Isn't grading students assignments and giving them grades in courses just this? Do we need another layer on top of this? Seems so, but I shake my head in disbelief at the money we will waste and the people's time we will waste to add this extra layer which in the end won't make us any surer of the output than we are now. Sorry to ramble and ramble..... but putting this down helped me encode it better. What do you think? (I bet Achim is listening) Mr. James B. Greenberg Director Teaching, Learning and Technology Center Milne Library SUNY College at Oneonta Oneonta, New York 13820 email: [log in to unmask] phone: 607-436-2701 fax: 607-436-3081 "Ignorance is curable, stupidity lasts forever"