Bob,

Very well said! As one of the others who voted in favor of the proposal
I believe you have effectively addressed many of the salient issues
related to the discussion of assessment. Your balanced analysis of the
issues surrounding assessment is a breath of fresh air in the
discussion.  Personally, I suspect that some faculty, especially those
who are most vocal in the assessment discussion, are being less than
honest when they say that they are not "opposed" to assessment but then
cite a litany of "anti-assessment" statements.  Thanks.

Bill 

Dr. William R. Proulx, PhD, RD
Associate Professor and Chair
Department of Human Ecology
100 Human Ecology Building
State University of New York 
College at Oneonta
Oneonta, NY 13820
Tel:607-436-2705
Fax:607-436-2051


-----Original Message-----
From: Teaching Breakfast List [mailto:[log in to unmask]] On
Behalf Of Rothenberg, Robert ([log in to unmask])
Sent: Wednesday, April 28, 2004 11:15 AM
To: [log in to unmask]
Subject: Re: Assessment and SUNY


Hi Jim,

I tried sending this yesterday, and had a lot of trouble.  I never got a
copy, so I assume it did not go out.

I tried sending it again this morning and got a message that it was
already sent, and I needed to revise it to send it again.

So, here is a revision.  If you already received it, please ignore this
copy.

Bob

Hi Jim,

As one of the few who voted in favor of the proposal, I want to add my 2
cents to the discussion.  I have not had a chance to see read the
material you attached, but feel strongly about this issue.

First, and most important, as faculty we must not be opposed to
assessment!  We insist on assessing our students; those for whom we work
should be allowed, in fact encouraged, to assess us.  If we are failing
in our work, we should have consequences.  BUT, assessing us should be
based on a clear understanding of what we do.

This starts with a definition of what is the institution's goal.  John
Carver and his wife Miriam have written two books on not-for-profit
governance (the names of which momentarily escape me) in which they
focus on three questions: what good, for whom, at what cost!  (Peter
Drucker has also defined the not-for-profit governance process in
similar terms.)  The key in their writings is that one must focus on
desired outcomes for (in this case) students, not on what faculty do.
The first step, then, in assessment is to define the outcomes for the
student.  This will have the effect of making certain courses more
similar than if we all did what we wanted to do.  Whether this
similarity is in the area of process (critical thinking, writing, etc.)
or in the content, or both, is something up for discuussion.

How much these outcomes will differ from institution to institution is
debatable as well.  Certainly certain content will not differ.  Students
from all institutions will need to know the answer to 2+2, and they will
need to vcome to the same answer.  Other content areas may differ
according to instutional emphases.  Process also should be fairly
similar.  Problem solving  is something all graduates should be capable
of - while they may have learned it in different contexts, the basic
approaches should not be too different. 

Once we agree on outcomes, it is possible to assess whether they are
being met.  Assessment should be built around the outcomes desired.  To
the extent these are the same, multi-institutional aassessment is
possible; to the extent these differ, it may not be possible.  I have no
fear about "teaching to the test;" in fact, it is the reverse: we should
be testing what is being taught.  If faculty are not engaged in
fostering the desired outcomes, then testing on them will result in
"poor grades" for faculty and/or institutions.  Maybe this is a good
thing.  

As to the impact of tests narrowing the curriculum, this only happens if
the testing is the "tail that wags the dog."  As curricular changes are
made, it becomes necessary to redefine the test instrument.  We recently
decided that my Law course needed to cover additional materials; when
the course changed, my exams also had to change.  If assessment need not
cover all outcomes, then maybe the tests do not need to change unless
outcomes undergo radical changes.

I think the greatest problem in this area is lack of trust.   Many
faculty do not trust students and vice-versa.  Many faculty do not trust
administrators and vice-versa.  System people do not trust institutional
people and vice versa.  Why this lack of trust exists and what can be
done to change it is a whold different discussion (perhaps more than
one).  

Second, why we should support (have supported) the proposal presented at
the Senate.  As I understood Aachim's messages throughout the semester -
assessment is coming no matter what we do.  The opriginal porposal from
the Trustees would have mandated a number of things regardless of our
concerns (valid or otherwise).  The most recent proposal allowed us some
possibility of meaningful input.  This latter is better than the former.
By rejecting all formns of assessment, we invite the Trustees to
categorize us as being "out of control" and this is against our
interests (and also those of our students).

Third: some specific thought about your comments: 

You make the comment: "One of  the most educated persons I know in this
community is the least tolerant  person I've ever met.  He continually
writes  letters to the editor of our  local newspaper saying how
tolerance will be the death of our way of life.   Would his college say
he was successful or not?  Would they say his  ideas are the desired
outcome of their programs?"  Perhaps the desired outcome is his ability
to engange in exchange of thoughts with the community.

You make the comment: "#5  hits  at the real burr I have about
assessment.  Isn't multi-level  assessment  what we do now?  Isn't
grading students assignments and giving them  grades in courses just
this?  Do we need another layer on top of
this?    Seems so, but I shake my head in disbelief at the money we will
waste and  the people's time we will waste to add this extra layer which
in the  end  won't make us any surer of the output than we are now."  I
think this goes right to the issue of trust.  Our students assess
whether they have mastered the material.  They regulate their study by
this means.  We independently assess them as a control on their self
assessment.  Why shouldn't our empoloyers (and funders) exercise the
same rights.  We all tend to become complacent, sloppy, ...  We need
outside assessment to make sure we do not lose focus. The money will
only be wasted if the additional layer fails to make us any surer; this
in turn will occur if the additional layer is not well thought out
and/or irrelevant.  

Lest this is misinterpreted, let me add that I share the concern that
any assessment imposed will be poorly designed.  However, I believe this
fight needs to be about that issue, and not about whether there will be
central and external assessment.

Is this a discussion for a future breakfast group?

Bob

-----Original Message-----
From: Teaching Breakfast List [mailto:[log in to unmask]]On
Behalf Of Greenberg, James ([log in to unmask])
Sent: Monday, April 26, 2004 8:29 AM
To: [log in to unmask]
Subject: Assessment and SUNY


Attached to this posting please find a PDF file by Shavelson on
Responding Responsible to the Frenzy to Assess Learning in Higher
Education.


Can you tell me what you think of this?

To get the ball rolling, I tell you what I think (it is quite a
ramble... sorry). I'm no expert in this area, in fact I feel like I
shouldn't even participate in the conversation at times since I know so
little... but what the heck, here goes...

At  last weeks Senate meeting the almost unanimous voice of the Senate
was to reject the idea of outside assessment as I understand it.  Every
argument that I can  remember from this meeting  is articulately
discussed in this
article.   Here are a few of those arguments that I remember and my take
on
them.

1- "Assessment often tries to measure  filling the bucket, but what
about lighting the candle?  Aren't we about  both?"  Shavelson discusses
this concept  with the notion of  higher learning vs. domain specific
knowledge.   We really need both  and are about both.  Most people
(surveys
suggest)  say that what  Higher Ed. should be about are the "lighting
the candle" types of things, but  assessment only hits at the filling
the bucket.  This is one of the issues that is the source of the tension
in this whole assessment argument if  you ask me.

Makes sense - most  of what I learned in College that I retained as far
as knowledge goes is in  Geography.  The rest of what I got from College
is the higher level  stuff, how to learn, sense of self, awareness of
world, etc. This  higher level stuff is what college goal statements are
usually about, but it  is the lower level stuff that  is usually
assessed.  The point about this being a problem with current assessment
practice is an excellent one and  that at the very least any assessment
we accept should make a clear distinction between achievements in domain
specific knowledge and more general abilities.

 2- "We all know that a standard  test, no matter how good, will result
in
 teaching to the test and narrowing  down the curriculum."   Shavelson
 discusses this issue as well.   Standard tests tend to move education
toward a narrowly defined  curricula - a one size fits all notion - and
away from the things we perhaps  TRULY value in our educational systems.
Things like sense of maturity,  ability to get along with people
different from self, etc.  A  real fear  faculty have (and I completely
agree with them on
this one) is that   standardized assessment will lead to a more narrow
curricula - which in turn  we will regret down the road.  We will, with
our honest good intentions,  undo  what we treasure most in our higher
educational system.

 3-  "I'm highly skeptical that any test or tests, no matter how good,
can  really  lead to good assessment and a process that betters
education."  Shavelson  does a nice job with this issue - giving the
history of  standardized tests and  the thinking behind them.  What they
are good for
and  what they are not  good for.   For example, do we want to measure
only
the  relatively  permanent knowledge students get, which we know comes
from  extensive  engagement, practice and feedback like the Pennsylvania
Study tried  to do or  do we want a more comprehensive assessment that
also tries to  measure  reasoning, decision making, etc.  Or, are we
interested in measuring domain  acquired knowledge (why does water make
a glacier move?). These  various  levels of knowledge are the problem.
Some are easier than
others to   assess.  Some, (the most important ones perhaps) may be
nearly
impossible  to  assess since only a lifetime of experience can bear them
out.    Perhaps a  survey of alumni over 50 is the best way to assess
these.

 In conclusion:   His propositions at the end  are good, but lack any
help.
 For example, I completely agree with #1  where we need to assess both
cognitive and social - civic things and we have  only been able to focus
on  cognitive up to this point.  But how do you  assess the other?  I'm
not  pessimistic by nature, but good luck with  doing this.  Assessing
social  things depends on values, culture and  morals.  If we don't all
share the same  ones - and I'm afraid our society  doesn't, then what
are good and bad outcomes?  An illustration, that  might not be
necessary to you is, "tolerance  of others opinions."  One of  the most
educated persons I know in this  community is the least tolerant  person
I've ever met.  He continually writes  letters to the editor of our
local newspaper saying how
tolerance will be the death of our way of life.   Would his college say
he
was successful or not?  Would they say his  ideas are the desired
outcome of their programs?

 His #2 is right on the money.  We  need to hold this conversation -
honestly  and openly with the public and  trustees and any others in our
society that  want to join in.  The  environment right now seems fairly
toxic to me for such  discussions to take  place.  Why?  Why can't we
invite the trustees and others  to "town  house" discussions about these
issues and also hear what they have  to say.

 #3 I've already spoken to.  We will narrow our curricula  and we may
not like  what we do to a wonderful system that is working pretty  darn
well at the  moment.  If people are worried about the growing costs  of
higher education  than let's talk about that.  But we shouldn't use
assessment to cut costs,  the price is too high.

 #4 is right on the  money and I would vote for using his conceptual
framework.   It is as good  as any I've seen.  (At least I understand
his!!!)

 #5  hits  at the real burr I have about assessment.  Isn't multi-level
assessment  what we do now?  Isn't grading students assignments and
giving  them  grades in courses just this?  Do we need another layer on
top of
this?    Seems so, but I shake my head in disbelief at the money we will
waste and  the people's time we will waste to add this extra layer which
in the  end  won't make us any surer of the output than we are now.

Sorry  to ramble and ramble..... but putting this down helped me encode
it
better.  What do you think?   (I bet Achim is listening)

Mr. James B. Greenberg
Director Teaching, Learning and Technology Center
Milne Library
SUNY College at Oneonta
Oneonta, New York 13820

email: [log in to unmask]
phone: 607-436-2701
fax:   607-436-3081

"Ignorance is curable, stupidity lasts forever"