Drop everything and go read GayProf’s latest, greatest takedown of Big Midwestern U’s new on-line (on-line!) student evaluation clusterfrack. Here are some tasty tidbits to whet your appetite:
The higher administration decided that moving the system to an on-line format would be so much better. Why? Well, because – um … It means that – er . . . It will just be better! It’s on-line!
Actually, they did offer us a set of fairly laughable justifications. One was that students must want to fill out evaluations on-line if there are unofficial sites like the much reviled (and often slanderous) Rate Your Professors. The logic being that the driving force of RYP wasn’t student entitlement or a means to trash instructors who required their students to work hard. Rather, it was simply the on-line format that kept students coming back. If they could fill out official evaluations on-line, than their desire for venting about their professors at RYP would be sated. It’s on-line!
. . . . .
So, has the university learned a valuable lesson from this colossal failure? Not at all – They have placed the blame on faculty for “not encouraging” students to fill out these on-line forms. If we really cared about evaluations, they claim, we would have made filling out this on-line evaluation an official assignment.
. . . . .
I hear you asking, “Why does any of this matter?” and “When is this blog going to be about gay porn again?” Both of those are fair questions.
Don’t get too comfortable, suckahs. GayProf predicts that if your uni hasn’t converted to on-line student evals, it will soon enough. After all, it’s all computerized and scientific and all that stuff.
Historiann 
We tried online assessments one year and one year only. It was such a colossal failure that the university administration abandoned the practice without a fight. Students didn’t use the university email and missed the requests or responded for only one of their five courses. Some also mixed up their course evaluations, leading to tirades against Professor A in an evaluation of Professor B’s teaching.
It’s been six or seven years now and no one’s talking online assessment. Here’s hoping that part of the institutional memory remains strong!
LikeLike
Thanks for the shout-out.
Like Janice, there were other problems with this system. One of my classes had a film lab. For mysterious reasons, that lab also got an on-line evaluation (even though the only things students did was sit and watch movies for zero (0) credits). It’s unclear if those students also filled out the regular course evaluation or if they were confused and thought the lab was the actual course.
Unlike Janice, I don’t see the on-line system going away any time soon. The administration is really loathe to admit failure with such a big idea. Besides, it’s on-line!
LikeLike
We did the exact same thing last year and it was a disaster. Only about a quarter to a third of students actually took the time to go online and fill them out, which spooked the t-t people looking ahead to the PT process — how will these evals be weighted if participation falls off so precipitously?
Some profs actually took class time to walk their classes to computer labs for the eval process, and never left the room while they filled them out. Interesting. And oh yes, others did indeed make filling them out a class assignment (and I know of at least one other university that does the same), which brings up all kinds of ethical questions. Given that class time is pretty precious (especially those of us in cold climes with snow days), I refused to take my students to a lab. Ain’t in my job description, folks.
We went back to paper evals this year, but the memo just came out — back online FOR GOOD next year. And you can bet any failure of the system won’t be due to the admin downstairs … it’ll be the faculty who somehow, someway aren’t doing their cheerleading best to get students in front of computer screens.
Once again, boobs who spend no time in classrooms making decisions for people who spend their entire career in them. Same bleep, different day.
LikeLike
Teh stoopid…it BURNS!
On the other hand, as I suggested in my comments on GayProf’s original post, maybe this shift to on-line evals will hasten their demise when everyone realizes what little value they have. This is one good side-effect of academia’s modal upper-middle class values: administrators can administer these pointless smiley-face surveys to students, who can fill them out, and the faculty whose job it is to interpret the data can just roll their eyes and toss the results into the trash. Everyone’s happy!
LikeLike
The computer people at our u. decided to go all electronic on evals (through our Blackboard equivalent) without having any discussions with faculty or chairs or even associate deans. When the new system came into place, response rates were really low. Surprise! And then the associate dean had to make the case to the faculty that we needed to remind, coerce, and threaten students into filling out the evals. The associate dean wanted to scream at and threaten the computer people, but he backed off because 1) everything else is going on-line, including daily newspapers and intimate relationships and 2) The trees that gave their lives for the paper forms.
LikeLike
Rad, your comment (and GayProf’s and Eduardo’s) raises an interesting question:
If students don’t want to fill out an evaluation, WHY ARE WE FORCING THEM TO? If the “customers” are perfectly “satisfied” (or satisfied enough) with the “product” they’re “buying,” then why hound them to fill out such imperfect instruments?
This goes back to my original conspiracy theory about evaluations: they’re just a means of collecting data that can be used against faculty at any point. After all, I’ve never yet seen a tenure case that was saved by student evals, but boy howdy, those evals sure are trotted out and waved around when someone wants to complain about someone else’s job performance.
LikeLike
That’s the key — if students aren’t interested in a particular assignment, they won’t do it. This is especially true for non-graded work, such as “you can derive this example for yourself when you get home” (who ever does those!) or “fill out this evaluation.” If the students aren’t sitting there in front of you, they aren’t going to do the work. Only a small subset of the class will be strongly invested in providing feedback — the students who were very impressed, or very angry (and very angry is more likely to provide feedback than very happy).
Frankly, the only way I can think of to require feedback would be to either prevent a final grade from being assigned (or to prevent the student from registering for the next semester) until the evaluation form is filled out. That’s not really in anyone’s best interest… although it could be a brilliant way of letting students avoid an F 😀
LikeLike
At Queer the Turtle U, we’re in the “remind, coerce, and threaten” students phase that Rad Reader describes. Electronic evals are, imho, an unmitigated disaster. Response rates are low, and those with axes to grind are far more likely to respond, which means their negativity is amplified. As we see so often in online communities, the rhetoric tends toward extremes of love and hate, and the hate gets directed at women, queers, racial minorities, those who lack job security, and anyone with reasonably high standards. The profs of Roxie’s World wholeheartedly subscribe to your conspiracy theory on evals, Historiann. They have never managed to save a career, but you’re darn tootin’ they have helped to sink many.
LikeLike
Our regular evaluations are still on paper, but as a I discovered last semester, the Honors College has its own online system. Other than asking the questions through a Web form, rather than a bubble sheet, the two appear to be identical. Teh stoopid comes in when they process the results via computer. (Both regular and honors evaluations appear to be processes the same way.) Evey question (except the free response comments at the end) is supposed to be answered on a scale of 1-5, and the results sheet they send to us has the overall average score prominently located at the top. Which might be useful, if the 1-5 score meant the same thing on every question! On about half the questions, 1 is poor and 5 is excellent. But there are questions about time investment, where no number is supposed to be best, and if a student thought the class was neither too easy nor too hard, they were supposed to give it a 3 on the corresponding question.
LikeLike
Roxie, somehow I knew that we were BFFs on this issue…
Buzz, your story takes the cake for teh stoopid–and once again, we see the total folly of student course evaluations. Peer review is not completely perfect, but at least it guarantees (or at least provides more assurance) that there’s someone in the room assessing teaching who doesn’t have an axe to grind. While I will readily admit that there could be abuses of this system–in the event a malign senior colleague uses it as an opportunity to shank a junior person–there will be other senior colleagues who will review the teaching, too.
Erica, you’re right that withholding grades would be one way to ensure compliance, but do we really want to be evaluated by people who are being harrassed into it? I sure don’t. Why not just let them fill out the on-line assessment forms after they get their grades? My guess is that they wouldn’t be all that different after the grades are filed, but even so…
LikeLike
Don’t worry — I’m not advocating required compliance in that fashion! 😆 It’s a bad idea; even if it ensures reviews from all (most?) students, it generates far too many other problems.
LikeLike
We’re just going over to the online evals in the coming semester… after it being postponed because of faculty outcry. The word in these parts is that we “have” to do the evals online because nobody makes the machines that read the paper ones anymore! If we don’t change now, what will we do when the machines break?!?@! Whatever the case, I’m glad I’m now securely tenured (unless something crazy happens between now and the BoT vote), as I won’t have to worry about how they will affect me. My plan is just to make up my own end-of-semester eval for each of my classes from now on and to ignore the online thing: a) I’ll get to ask questions that actually matter to me, which will actually give me answers that help to improve my teaching, and b) I will no longer have to pretend I understand what 4.74 means about my teaching 🙂
LikeLike
Historiann, your post reminded me that we hadn’t gotten our evals back yet – you know since last semester was the first time going to online evals. I can’t imagine why we haven’t gotten them?! (puts hands on her cheeks in an “oh my” look). Of course, I’ve only been here 18 months but no one has once asked me if my evals have been ok. They could care less. If your teaching stinks, you sink, but otherwise it’s research, research, research and the students don’t matter.
LikeLike
The only evaluations I ever really liked were 3:
1. Department where I was a T.A. 120 questions!!! They asked things like, could the instructor be heard? When they wrote on the board, was it legible … or did they write so lightly (in chalk) that you couldn’t see the letters from the back of the room? Really practical things. Quite useful, especially for first time teachers like me.
2. Next department where I was a T.A.: essay evaluation. No instructions beyond: “please comment on this course.” That forced people to be coherent.
3. In Brazil. Last day of class was evaluation day and students, in front of everyone, gave their views directly to the professor (if they wanted to), and the professor responded on the spot. I was not warned this was going to happen and couldn’t believe it when it started – mayhem will break loose, I thought – but no, it was coherent, civilized, and even useful.
LikeLike
Prof. Zero–those are examples of good questions that will elicit useful information (in #1). I got useful feedback from students when I taught small classes at private universities, and learned that (this seems obvious, but for example) they would appreciate a lecture outline at the beginning of class. I had never been offered a lecture outline as a student (u/g or grad), and always took good notes, but didn’t realize that there were other students who couldn’t see the logic of the lecture unless it was signaled in advance. That was helpful. The editorial comments to the effect of, “who the hell do you think you are?” that I got then and still get now–not so much.
The Brazilian model seems a touch too coercive even for me, but I like the notion that people will be held accountable for their comments. That’s the flaw with the current eval system now–they’re anonymous, whereas in real life, when you provide feedback to a boss, an employee, or a student (for example), you have to do it constructively and leave out the ad hominem attacks.
I think it’s hillarious that these new, scientific, and computerized evals take longer to calculate and distribute than the old scan-tron sheets, as Dr. Crazy and Liz2 report. Everything old is new again!
LikeLike
As I read this post, I tried to put myself in the position of a student here, and what I came up with was distressing: the only way I, as a student, would take time to fill out an online evaluation outside of class time would be if a) I really, really liked the professor; or b) if I really, really hated the professor. It’s the same logic that drives RMP, which is what makes it so utterly useless as an assessment tool.
Plus, I second GayProf’s original sentiment. It seems that if you want to sell an administrator on anything, just affix “e-“; “cyber-“; or “online” to it.
We should be able to find a way use this to our own nefarious ends, no?
LikeLike
Indeed we should, Notorious, but most of us are basically kindly and earnest dullards who really don’t get it that anyone would want to sandbag us or sabotage our work.
I’m working on getting over that earnestness myself. This blog is a big help!
LikeLike
Hi Historiann and co.,
I appreciate the thread. I am a little chagrined to report that Whoville State University, where I teach does not have student teaching evaluations. We are encouraged to preform our own, but there is no institutional mandate or form. We’ll see if this survives the latest round of union negotiations.
That said, I do my own evaluations at the end of the term, like Dr. Crazy. The best part is I control the information, and it is useful to me and the students. I get to ask the questions I want. I can voluntarily share the (relevant) results with the colleagues and the dean. I have made lots of changes to my courses, based on student responses to the evaluations. Its worked out better than the institutional evals I had to do as a TA & Grad Instructor at the University of Megalomania.
LikeLike
Is it really that hard to imagine that making student evaluations available through Facebook, placing participants into a drawing for season football tickets or free textbooks at the bookstore, and making the aggregate results of all student evaluations available on the student government web site are all feasible and starting to happen at some campuses?
LikeLike
PhD in History–those incentives might work to elicit more responses, but they wouldn’t seem to encourage very professional or useful responses. Are there such programs at universities now? Please, fill us in and don’t be coy.
My university has a website hosted by student government that features volunteer evaluations, but those suffer the same problmes as the Rate My Professor website–it’s only the really happy or the truly disgruntled who respond.
LikeLike
You know, it occurs to me that before we can evaluate the feasibility of doing evals online, we need to know what they’re for. If they’re to give students a voice, help students with figuring out which profs they’d like to take, etc., then perhaps what PhDinHistory describes is an improvement on the current system. If they’re supposed to me a measure for evaluating faculty performance – and if they’re used to calculate things like merit pay and to determine whether a person has earned tenure – perhaps we need a method of delivery that reflects the seriousness of that? Because here’s the thing: I wouldn’t want my merit pay (or god forbid, tenure) to depend on my students being jazzed to win football tickets. What if I’ve got a class filled with people who look with disdain on such prizes (and really, students I teach look with disdain on most things, which may have something to do with the sorts of students who gravitate toward my discipline generally and the kinds of courses I teach specifically)? Or what if they don’t give substantive feedback, but rather just write any old thing in order to get their name in the raffle? Is that really useful information?
Finally, and this is the problem that most of my colleagues had when we discussed the move to online evaluations: we’re all convinced that our students will only fill them out when they are drunk or stoned. This may say more about what kind of students we ourselves were than about our students. It also fails to note that it could well be the case that students could be drunk or stoned in class on evaluation day as well….
LikeLike
Exactly, Dr. C.–the evals at my university are meant to be used by T & P committees and department chairs in evaluating faculty performance and, in part, in deciding annual raises. (In the event there is a raise in the works–there won’t be this year.) “Incentivizing” students by entering them in raffles for football tickets and new pickup trucks just seems to feed the consumer-oriented beast that demands infotainment rather than education.
LikeLike
One of my friends just appeared in the student newspaper at my institution because he has high student ratings and history of giving lots of A’s, according to Pick a Prof.
Rate My Professors is already an app on FB. This allows FB users to access student-written reviews of professors and their grade histories.
There are a few reasons why I am glad to see these developments. I think students have a right to know in advance whether the prof they are thinking of taking a class from is a jerk. When everyone has access to every prof’s grading histories, via Pick a Prof, it can create pressure for the hard and easy instructors to move toward the average when it comes to assigning grades. The reviewing process online also creates a way for students to hold professors accountable for providing quality teaching.
I agree that you all have good points about how these kinds of online web sites can encourage the consumer-mindset of our students. And students are certainly not always the best judge of how much and whether they have learned things during the semester. One simple way to solve this dilemma might be for our departments to design a pre- and post-test for the freshmen survey classes. The students would be told that the test was part of their grade and the results would provide a useful measure of how much the students were learning over the course of the semester, as well as how different sections taught by different instructors compared. These department-provided tests could count for less than 5% of the overall grade, so professors wouldn’t have to feel like they were teaching to the test. In our 200, 300, and 400 level classes, I think students should be offered extra credit if they will keep a typed weekly journal that briefly tracks the teaching in their course and then submit it along with a one- or two-page report to the department chair or dean at the end of the semester. I think we should take the initiative in coming up with innovative ways to obtain teaching/course evaluations, before standards and practices emerge with new technologies that could shut out our input.
LikeLike
Thanks for the link, Ph.D. in History, but I have to say that it’s not surprising that the top-rated teachers have what looks to me like an extremely suspicious grading history of giving As to 52-73% of their students! They may in fact also be fine teachers, but this just confirms my suspicions and those of all of the other commenters here about the fatuity of student evaluations in general, but of these facebook/myspace/Infotainment-app friendly incarnations in particular.
This ain’t Lake Woebegon. All of the children are NOT above average.
LikeLike
Great post, Historiann. About your comment @1:40: I’m not sure the online evals would go away even if they’re shown to be useless, because administrators are interested in numbers and data. Usefulness is beside the point.
LikeLike
Pingback: Friday round-up: we ain’t got the do-re-mi : Historiann : History and sexual politics, 1492 to the present
Pingback: Privatization: what could possibly go wrong? : Historiann : History and sexual politics, 1492 to the present
Pingback: Assess-mints, now with extra extra pointlessness! : Historiann : History and sexual politics, 1492 to the present