Oooh, Bardiac–don’t “despair!” Have an assess-mint–they’re new and improved, with extra extra pointlessness!
Friends, Bardiac is being threatened to perform an assessment exercise, because you know that if we don’t do it, the “they’ll do it to us” boogeyman will jump out of the recycling bin in the hallway and we’ll regret it! (We will!) So (the argument goes) we should just do extra work because someone has issued a threat. But Bardiac is skeptical:
I have to say, I’m so sick of the threat that “if we don’t do it, they’ll do it to us” that I could spit. The they of that threat isn’t doing diddly that’s of use. There’s no serious evaluative stuff happening here, despite blathering and spending money up the wazoo to talk about it. And what is being done is being done to us already, in ways that make me lose a class day in intro writing every fall.
I’ve never yet seen anything useful come out of all the money they’ve spent and all the crap they’ve made us do. I have seen vicious nastiness happen, however. And I’ve seen threats of more to come.
Do any of you have experience with the “they’ll do it to us” boogeyman? Like Bardiac I’m skeptical, but I’ve got an open mind if you have a tale of woe to share. I’d like to hear all about it. Don’t forget to leave your collective wisdom in the comments at Bardiac’s place! (Otherwise, “they’ll” do it for you, and you won’t like that now, will you, missy?)
Historiann 
I think the “they’ll do it to us” thing is a really crappy argument in favor of assessment and typically results in assessment strategies that aren’t terribly well thought out or useful. I actually sat through an assessment training thingie with an “expert” (who had actually been a faculty member! with a PhD in a discipline and not in some Ed variation thereof!) and I feel like the way assessment was explained and proposed in that training was about as reasonable as anything I’ve ever heard. The gist was this: assessment should be, for the most part, invisible. Students shouldn’t be aware that it’s happening; faculty workloads should not increase excessively, and, in fact, unless a person was on an assessment committee, faculty workloads wouldn’t increase *at all.*
The problem, of course, is that even though my university brought this person in as an “expert” (and even though this person has been responsible for getting assessment up and running at various institutions across the country), none of the upper administration wants to follow this person’s advice (or to invest in the institution and in faculty training so that it can be followed). And so, cumbersome assessment that is nearly impossible to implement. As you would say, H., *awesome*!
Somewhat off topic, the people who like to throw out the “they’ll do it to us!” bogeyman also seem to be the same people who like to assert that any new policy or practice is “a lawsuit waiting to happen!” Note: none of these people have any background in a) assessment, b) the law, or c) anything that would make their opinions worth listening to.
LikeLike
Amen.
The gaping maw of assessment is consuming all serious intellectual activity at colleges and universities across America. And I despise the disingenuousness of the “they’ll do it to us!” crowd–which is generally made up of people who are directly profiting by getting on the Assessment Train, but are too cowardly, or just simply incapable, of defending these bureaucratic processes that attempt to reduce learning to numbers.
The tragedy is that most professors have been doing meaningful, intimate assessment their whole careers–they think deeply about the assignments they create, read and evaluate student work, and reflect on what their students are and are not learning. They engage in conversations with respected peers about ways to improve their teaching and enhance student learning. The more time spent writing assessment plans, post-assessment reports, and counting the most trivial things, all to appease an assessment industry growing more powerful and self-serving every day, is the less time faculty spend on meaningful evaluation, reflection and innovation. When will we awake from this anti-intellectual, bean-counting nightmare?
LikeLike
I agree with Dr. Crazy. Our assessment person said basically the same thing. You shouldn’t have to do any extra assignments or projects for a Department level assessment plan. What ever the standard is in your discipline should be the standard for the assessment.
So Bardiac’s first response was right. Spend an hour with the colleagues norming the rubric, use it on your papers from one or two sections that semester and you send it to Institutional Research to be counted. Done. Anything above and beyond that is:
a) pointless
b) won’t get done
c) waste of time
Sometimes assessment is worth doing. Our English Department did a similar Assessment of five sections of compositions at Woebegone State last year. This semester dean used their assessment information to save their smaller class sizes and fixed term faculty during the budget cuts. (Mind you we’ll have to wait and see if that trick works next year…)
LikeLike
I actually have some sympathy with the “They’ll do it to us” argument, as assessment is part of accreditation these days. That doesn’t make it right, but it is part of the new reality whether we like it or not.
That said, this notion of invisible assessment fits in very well with my developing philosophy of university administration. Here it goes: Their job is to keep us from having to do useless things like assessment so that we can do our job, namely teaching students and other professorial activities.
An invisible assessment would likely require very little extra work than is already being done, which is a good thing. Bardiac’s administrators have clearly failed her on this front. The need to make threats though is the most obvious manifestation of their ineffectiveness.
LikeLike
At our place, the upper adminosphere (populated by Ed.D.s) wanted us to implement a distance learning program that our department found entirely questionable and cynical. The adminosphere asked us to discuss the plan in our department. We did, and said “no thank you, we’ll do it our way.” They asked us to discuss it again. We said “no, but here is our alternative.” They ignored our alternative and asked us to discuss it yet again. Yet again, we said, “no.” Then they came down with “do it or we will do it to you.” We asked, “how, exactly, will you do this to us?” We really wanted to know since we were all the experts in the field and the only other option that they seemed think of was to purchase software from a company, and they sure didn’t want to do that. We also suggested that the union might be interested in the whole procedure. Suddenly, the upper adminosphere went from “do it or we’ll do it to you” to “oh, no one is being required to do this! We just thought it might be a good idea.”
Score one for solidarity.
LikeLike
Can anybody out there excavate a YouTube clip of the famous “Reagan pod” episode(s) from Saturday Night Live, c. 1979? Cause that’s what the “they’ll do it to/for you” exercise is all about. They find people they can “flip” inside an institution and pretty soon this subversive narrative gets going about what the black helicopter folks will do/have done somewhere else when faculty get obstructive. Then somebody else flips, then somebody else flips, then it’s all playing dee-fense.
The fact that it’s connected to accreditation seems to me all the more reason to look into overthrowing accreditation itself. Where do these accreditors live when they’re not dropping in to announce what they “need to see” the next time they drop in? Does anybody live near an “Accreditation Tower” in some actual city, where folks like you and me come and go days doing excellent work? Who accredits the accreditors, and what defense do we or anybody have against whatever hot theories pop up in their hot tubs or webinars? With the world going crazy from Tunis to Madison, maybe it’s time for some transparency therapy on Accreditation Row, followed by Assessment Alley. It’s all about what academics like to call relations of power, in an industry where aversion to power has long been a core principle, or at least practice. I think that principle/practice is what the real underlying target is in much of this issue.
As for whether it takes extra time, it doesn’t here. We just make the $h^+ up. But if there was ever a lawsuit waiting to happen, that’s gotta be it I would think.
LikeLike
What is “assessment”? I get that it’s something where you try to figure out if students are learning, but what is it operationally?
LikeLike
I think what is needed is a little middle management Jujitsu, can somebody demand that the administration assess their assessment tools to see if they are impactful in any operational way ?
I have taken the hold my breath and stamp my feet approach to such entreaties by the HR people that seem to run the very big hospital system into which we have absorbed. When asked about any deficiencies with the PA that works with us (Grandfather of 2, been doing this for twice as long as I have, steals patients from me regularly, has more heart than the rest of the deppartment combined)I wrote that any shortcomings would be discussed between him and myself, end of conversation. I talk a big game for now, maybe if they bring the hammer down I’ll just make stuff up as well, thanks for the insporation Indyanna.
LikeLike
Warning: long somewhat off topic rant. Skip to last paragraph for shorter version if not interested in K-12 stuff.
Over here in ritzy K-12 land, we talk a lot about assessment. In part because we give a lot more tests, papers, quizzes, and projects than you all do and it’s important for us to do it right if we aren’t a) going to lose kids and b) not going to have behavior nightmares in the classroom. (Side note: the biggest shift from teaching college to teaching HS was dealing with the fact that there lots of people in the room who didn’t want to be there and had to be there anyway, which didn’t really happen in my college teaching – they just didn’t show up).
So, in my department we have been phasing out multiple choice and nobody takes the SAT IIs in History anymore. Our curriculum is just too different from that of the US History test and our kids stink at multiple choice questions b/c they so rarely encounter them. We are examining portfolio grading. We give lots of brainstorming assignments, essay outlines, and a couple of essays and every test has some writing on it (even if it’s parts of an essay like “give three controlling ideas that support the following thesis”) Our short answer questions are based around specific history skills (cause and effect, “connections”, quote identification, etc.)
The killer is group projects. We keep hearing that colleges want kids that can do group projects and it’s important for us to teach kids how to do group work properly. This strikes me (and some colleagues) as a nightmare. It’s hard enough to design a course that teaches content and reading, writing, and thinking skills (not to mention provides an overview of how read different source types from traditional docs, to city plans, to archeological artifacts, to art, video, audio, etc.). But how to design a rubric for group projects that a) is fair and b) results in a semi-equitable distribution of labor. I don’t want kids making four films a year (a not very fancy film takes them about three days to make using cell phone cameras and/or flips and laptops). But there is a huge inequity between script writer, researcher, director, and editor jobs. The conventional teacher wisdom says you set up a group and they rotate jobs. But a) my classes are mostly 18 not 16 and b) four films a year? Blech.
And unified rubrics, don’t even get me started… I saw one recently that a school uses across all disciplines for all projects. yeah, that’ll work.
Shorter version: as the old joke goes, “and then a voice boomed out ‘You think you got problems, I sent my son to Israel to learn about the culture…”
LikeLike
Thanks for the nod, Historiann. And for the kind words, folks. It helps to know that other folks hear the same thing.
LikeLike
@Dave I am told that the reason colleges are supposed to assign group work is that corporations want people who can do group work.
I have a dark view of it. I think clubs and student government are good places to learn group work. I think a lot of the focus on groups doing projects has to do with adjunctification: if you have people teaching a lot of courses out of field, hired without a lot of lead time to prepare, you can’t expect them to seriously lecture and guide individual projects in all of these fields. So you have them work with the class on group research projects.
When *I* was taught as a freshman, adjuncts with that situation would give the basic lecture and send us to do library research and also to interview experts around campus. That was possible because there was a real library and real experts. These adjuncts were teaching us, in part, how to go and introduce yourself to a Major Person. It was useful. But now, elsewhere, I see some of where the need for group projects comes from.
I do have one group project I really like: put on a play, in the introduction to literature. I use short, modern, non realist or even anti-realist plays, so there aren’t a lot of props and costumes. They lend themselves to different interpretations and so each group has to put one on and explain why they’ve made the staging choices they have. It’s an exercise in interpretation and it gets everyone involved. But so many group projects are so unweildy and unfair, I don’t like them.
LikeLike
The assessment in our department all stems from our programs not completely passing our accreditation right before I got here. The accreditation agency required it, and it will be forever required by them until they’ve got their brain scan technology perfected.
Or, in other words, they did it to us. I don’t know if there were any other threats before this, however.
LikeLike
To tell you the truth, I have no idea how our assessment data is related to accreditation or program review. Maybe if someone explained it to us (or if I had a better memory) I could see some logic to it.
So much of this talk of assessment seems so defensive and unnecessarily pre-emptive, but for what? That’s why I liked Bardiac’s comments and highlighted the boogeyman talk. She’s not the only one hearing this scare-talk.
LikeLike
Oh God, here I go…. I have papers to grade, but I just can’t help myself….
Here at Five & Dime State, we’ve been given the exact same line by our administration: either we do it ourselves or it’ll be done for us. But that’s kind of superfluous since assessment is being imposed on us anyway by the Higher Learning Commission. (IMHO, accrediting agencies have been engaging in some severe mission creep. Instead of helping colleges “use assessment to develop a culture of continuous improvement,” etc., they ought to show up on campus just long enough to certify that we’re not a diploma mill and then GTFO.)
Maybe we should think of this as just the latest chapter in the long history of psychometrics-as-a-tool-of-technocratic-liberal-reformism. (Is there a German word for this?) In my discussions with members of the assessmentariat, I always ask if they’ve read The Mismeasure of Man or the Mismeasure of Woman. The answer is always, “Huh?” Which is to say these people have no idea of the broader history of what we’re doing (at least none that goes back before Margaret Spellings), much less of the possibility that our well-meant efforts might actually prove harmful.
A few of my stock comments on assessment:
1. It’s probably true that any good course (in, say, Intro to Lit) will help students “achieve” Student Learning Outcomes X, Y, and Z. But said course is never *reducible* to X, Y, and Z. (I always have to explain the concept of reductiveness to assessmentarians. I usually illustrate by saying that, even though every human being is composed of carbon, sodium, etc., not every combination of carbon, sodium, etc. is a human being. A fresh corpse will have the same chemical composition, yet it’s not exactly the same thing. Ditto for courses: some have life, some don’t.)
Thus two different Intro to Lit (or whatever) courses might not be of equal quality, even if they both assess exactly the same on SLOs X, Y, Z. One might be far richer and more rewarding to students than the other (which for all we know has been stripped down to “cover” only the SLOs.) But an “assessment instrument” geared only to the SLOs will not register that difference. Because, you know, it is reductive. This can be true not only of individual courses, but also of entire programs.
Of course, if you’re in the distance delivery business and want to maintain the self-serving fiction that “studies have shown that our online courses/programs are just as good as our on-campus courses/programs,” this is not exactly a disadvantage.
2. In the humanities, at least, there are often profound disagreements over just what the discipline is ultimately about. In English, quite famously, some see the discipline as about “passing on the Great Tradition,” others see literary study as a form of social critique, yet others see it as a way to develop advanced levels of literacy, etc. With such disagreement on the basics (which I think is a *good* thing!) agreement on SLOs can be very difficult to come by. Any resulting list will leave some things out that are extremely important to some profs, which means that pressuring them to “teach to the SLOs” is a de facto violation of academic freedom.
3. The current assessment mania is an instance of technocratic liberal reform. In Lionel Trilling’s introduction to The Liberal Imagination, he laid out the now familiar* argument that such reforms often go awry because, as they pass from desire to fulfillment, they necessarily become embodied in institutions, and those institutions–
a.) will inevitably prove better at realizing some aspects of the desired reform than oothers (e.g., those aspects which are numerically measurable will grow in importance, merely because they’re easily measurable); and
b.) will inevitably develop competing objectives of their own, not least of them the continual expansion of their own bureaucratic empires (which all of us are probably already seeing with assessment).
These tendencies can distort the original, laudable reformist vision beyond recognition, but fortunately (says Trilling) this regrettable fact of liberal reformism can be countered by those educated in the humanities, since one result of humanistic education is the ability to see the entirety, the whole, the historical situatedness, etc., of what modern life inevitably works to isolate and reduce.
Something like that. Trilling says it better. Read for yourself and see what you think. FWIW, this used to be a fairly standard neoconservative argument, back when conservatives had brains.
* Well, not so familiar to assessmentarians, who seem to operate wholly within the technocratic-liberal-reformist ideology. Critiques like Trilling’s are not only unfamiliar to them, but very, very difficult for them to wrap their brains around, being beyond their conceptual horizon and all.
4. Unemployment is pretty bad, so what the hell. Assessmentarians need jobs, too.
5. This too shall pass.
LikeLike
Pingback: Solidarity will save us all. « More or Less Bunk
@Fratguy–love your word “insporation”: it has just the right “Invasion of the Body Snatchers”/pod-people overtones.
LikeLike
RE: Assessment’s relationship to accreditation. Woebegone State has been under the hammer from HLC (like Eveningsun and wini) to set up a University-wide assessment plan for the last two or three cycles of accreditation. If you don’t have one, then you don’t get accredited. No accreditation no money from the Feds.
Mind you, Accrediting Agencies like HLC are authorized by nobody, responsible to nobody, but self anointed, with a nod from the head by the Department of Education. Mind you DoEd does not have oversight or accredit the accreditation agencies. They just wont give financial aid money to schools that are not accredited.
LikeLike
@ Eveningsun:
“Maybe we should think of this as just the latest chapter in the long history of psychometrics-as-a-tool-of-technocratic-liberal-reformism. (Is there a German word for this?)”
There is a German word for this, Gleichschaltung. It was quite popular between 1933 and 1945.
LikeLike
“Gleichschaltung”–that’s it! Thank you, Matt_L.
Wikipedia: “Gleichschaltung, meaning ‘coordination,’ ‘making the same,’ ‘bringing into line’ is a Nazi term for the process by which the Nazi regime successively established a system of totalitarian control and tight coordination over all aspects of society.”
“Bringing into line” is just another way of saying “alignment,” as in “Your course’s Student Learning Outcomes must be aligned with the General Education Objectives 1.C.ii, 3.A.iv, and 4.C.iii (or Program Objectives 2.A.vii and 3.C.i), which in turn must be aligned with the institutional mission, increase retention, decrease time-to-graduation, and help us fulfill our state Performance Contract.”
Of course, if the course under discussion is a required course in any teacher-education-related degree plan, those same SLOs must also be “aligned” with state K-12 “Content Standards.”
These various alignments are “assured” (in quotes, because in the classroom, teachers actually pretty much do whatever the hell they want, and thank God for that) by the creation of mind-numbingly detailed matrices that indicate which “content standard,” “program goal,” or “general education goal” is being addressed by which Student Learning Outcome, by means of which assignment on which day, etc., etc.
Historiann, if you haven’t yet served as your department’s liaison to the Teacher Ed department’s state reauthorization person for social studies, you might consider volunteering just so you can experience the sheer dystopian joy of it. Think Terry Gilliam’s Brazil turned into reality TV.
LikeLike
Well, as I noted over at Bardiac, we are doing it because WASC requires it.
I think most assessment is badly designed, and worse, it requires excessive documentation. Good assessment, as Dr. Crazy and others have noted, should fit seamlessly into the curriculum. That most assessment is badly designed etc does not mean assessment is bad. (Our accreditor says they respect qualitative data, but everything has to fit into a table. Go figure.) I think it is helpful in helping us connect the work done in different classes into an overall set of outcomes. That is, for years I thought about what I wanted my students to learn in my classes, but it did not necessarily connect with what my colleagues wanted them to learn. And assessment, in my experience, forces all us famously independent types to think collectively about what we are doing. (Some of you are in departments which do that already; that has not always been my experience.) I *Do* think that is good.
Since our Program Outcomes are skills based, we end up thinking after we’ve done an assessment about how we might collectively support students in getting this skill better.
THere are politicians who would like us all to do standardized tests at graduation, and the Collegiate Learning Assessment is a good one in that list. Look at the Spellings report. So the more we can say, yes we’re looking at this, and doing this, etc. the better. So the “if you don’t… or else” is partly true. Alas.
LikeLike
You know what? I am not sure it’s a bogeyman, and even if it isn’t, I’m just fine with that argument. Maybe it’s because I keep running into people who can’t be arsed, and I’m sick of it. The thing is, I really think that assessment stuff should be all the things that Dr. Crazy says. I think it *can* be that way if people are good colleagues who regularly talk about teaching with their peers, and who talk with their department colleagues about their program and don’t all think that academic freedom means that no one has a right to ask them what the hell it is that they do in their classes, or suggest that there be departmental norms (not requirements, but some sort of expected range/similar definition) for amounts of/types of reading, writing, exams, etc. At its best, all of this assessment stuff allows faculty to sit down and and talk about what students are learning, what they aren’t, and maybe (and here’s where the resistance is rooted, I think) ways of teaching that will help ensure that more students are learning what we say we teach.
Frankly, I think that just having conversations where faculty are forced to articulate to each other what we mean by X, and that when we assign students to do X, students understand what X is, and that’s X is X1, X2, X3, etc., depending on the faculty, but is still always recognizable as X, and never Y. Because to me, the “we’re all professionals and we know how to do our jobs and we know a good essay when we read it” argument is not enough. If we cannot be bothered to discuss these things with our colleagues — the other professionals — then I for one have no faith that we are explaining these things to our students. And at least at my campus, students aren’t professionals and often don’t know what a particular assignment entails, let alone a good version of that assignment.
I could rant more, but honestly, when I’ve worked with people who have bought into the idea of needing to do assessment because it’s part of the job, I’ve learnt a lot; the work, including some serious number-crunching AND qualitative evaluation, has been relatively easy and very collaborative, so not really that burdensome, and useful both as a way of touching base on student needs AND on justifying requests for resources. I’ve worked in the opposite environment as well. The results range from general stagnation to the truly toxic.
LikeLike