Well, if you teach in a Ph.D.-granting institution, what counts is research, and if you teach in a bachelor’s degree-only institution, it’s teaching. With apologies to Depeche Mode, not “Everything Counts, in Large Amounts.” This unsurprising result is brought to you by a national survey of Political Science departments, published in the most recent edition of PS: Political Science and Politics, and reported by Inside Higher Ed.
A national survey of department chairs found that superior research compensates for “mediocre teaching” at 55 percent of Ph.D. granting institutions, compared to 34 percent of master’s institutions and 17 percent of bachelor’s institutions. A contrasting split is evident at bachelor’s institutions — although many of them do not claim that their faculty are committed to research. At 64 percent of bachelor’s institutions, superior teaching would compensate for mediocre research, while that’s the case for 38 percent of master’s institutions and 14 percent of doctoral institutions.
. . . . . . . . . . . .
Departments from different sectors share some approaches to evaluating teaching. Overwhelming majorities across sectors report using teaching evaluations, teaching portfolios, syllabi, and peer review of teaching by other faculty members. But department chairs or deans are much more likely to be involved in peer review of teaching at bachelor’s institutions than doctoral universities. For instance, 69 percent of chairs reported doing peer review of teaching at bachelor’s institutions, compared to 27 percent at doctoral institutions. For peer review by deans or senior administrators, the figures were 31 percent for bachelor’s institutions and 3 percent at doctoral.
This survey appears to be a relatively blunt instrument, because I think the more interesting questions are: 1) What kind of teaching are we talking about: survey classes, upper-division elective courses, or master’s or Ph.D.-level seminars?, and 2) in any case, how do we know what is good teaching? The demands of constructing a lecture course for hundreds of students in a darkened lecture hall are totally different (and much higher, I think) than the demands of constructing a graduate course for cozy discussions for 8-15 graduate students, and both of these courses are totally different from putting together an effective research methods seminar for juniors and seniors. I tend to think good or bad teaching is more discernable in peer evaluations of teaching than in a review of syllabi and student evaluations only, and only by reviewing teaching across the curriculum that a given faculty member is expected to teach in.
(But then, this may be a self-interested thought: My evaluations always look like they were written by two entirely different classes, one of which hated me and the class, and the other of which really dug the class. Few people feel neutrally about me or my classes. I’m also the kind of professor who runs into a student several semesters or years later, and the student says, “gee, I thought you were totally unreasonable when I was in your class, but your class was where I learned the most/was the only one that prepared me for graduate school/was the only one where I was encouraged to think/etc,” which only leads me to question the value of student evaluations further. The value of our teaching may not be discerned by a student until years after they wrote their evaluations in a peevish or distracted state in May or December.)
How do you define good teaching, and how do you use those standards to rate yourself or others? I have to say that I’m extremely humble about evaluating other people’s teaching. (At least, I’ve never written a negative peer evaluation of anyone else’s teaching. When I observe others teaching, I always learn something about the subject matter at hand and about teaching itself, and I’m probably more forgiving than your average 19-year old who is distracted from the business at hand by romantic intrigue/a hangover/roommate troubles/text messaging/or all of the above.) What works for one faction of the student body won’t work for another, so shouldn’t we just let a thousand pedagogical flowers bloom? Moreover, in a department that produces Ph.D. students, it’s entirely appropriate that research is valued more, and that (according to the IHE article) “superior research compensates for ‘mediocre teaching?'” (Once again: what’s mediocre, in which context, and isn’t mediocrity in some contexts OK once in a while? Or are we supposed to be like the children of Lake Woebegone, “all above average,” all of the time?)
Historiann 
A few additions to your remarks about teaching evaluations (with which I largely agree): in my uni, it was universally acknowledged that the evals for large lecture classes tended to be lower than average. In addition, evals for *female* profs of large lecture classes were disproportionately low. The way students react to a professor result from a wide array of factors, including personal issues, gender, and structure of course. (I don’t think we can or should underestimate the role that the professor’s age, gender, perceived sexual orientation, and accent can play on students. Accents are esp. pernicious, since some accents are considered a sign of “intelligence” [ie, British man], whereas others a sign of being incomprehensible [Asian, African, etc.] I was astonished at how many remarks on my evaluations could only be described as *personal* – by which I mean they were directed less obviously by dissatisfaction with something about the course, but something about me, which led me to realize how deeply students’ reactions can be embedded in their own mental space at any given time. The only general use I can see for evaluations is to discern a problem with a faculty member if the numbers are consistently, disproportionately poor – that is, as a potential red flag.
The focus on the “numbers,” especially for tenure, presents several problems, many of which I’ve referenced above, or Historiann did in her remarks. I especially appreciated her remark about mediocrity – isn’t it okay sometimes? Particularly if one teaches in a department renowned on campus for having very high eval numbers. Somebody has to come in lower – low numbers don’t necessarily indicate a problem. To me, having a few mediocre teachers around isn’t a big deal. Effort is important. Teaching is a learned skill for many, while others seem to have a wildly charismatic presence that a broad array of students respond to immediately (I’ve noticed anecdotally that most of these individuals happen to be men). Besides, even if one IS a good teacher (and good teaching is important to me), sometimes a class just bombs, and it’s nobody’s fault. On top of that, of course, most of the best teachers have been teaching for many years; it’s not easy to pop out of grad school an amazing teacher.
I like the idea of peer evaluation, especially if it involves ongoing, rotating evaluators, because such people have the potential to provide useful feedback to the faculty member. I was evaluated each year in my first job, each time by a different faculty member. It was a positive experience.
IMO, good teaching begins with clarity of delivery, organization of ideas, & positive interactions/ engagement with the class. A higher level of teaching is reached when the teacher is able to help students forge connections between ideas, analyze primary material at a deeper level, and understand and improve written skills.
LikeLike
As you know, I really think student evaluations have been given too much weight in the past (Truth in advertising: My evaluations often look like yours: lopsided). Student feedback is important to refining our course content or approach, but I totally reject the notion that students are “customers who are always right.”
For me, I think good teaching would be about giving students new skills or methods for thinking through problems. So I would want history courses to focus less on spewing out names and dates (something students hate anyway) and focus instead on training students to think about the past critically.
LikeLike
GayProf–yes, exactly. But, I have had surprising resistance to the “think about the past critically” thing. (It’s pretty much all I do, at every level of the curriculum!) Many students just want a survey course (especially) to be facts and dates they can extract and regurgitate from a textbook, and they’re angered by the concept of having to do higher-level reading, thinking, and writing.
Of course, I warn them to DROP OUT! DROP OUT OF MY CLASS NOW! if this isn’t their style. But do they listen to Historiann? No. They stick around, find themselves stuck, and then take it out in their course evals, which is why I think student evals are teh suck.
As perpetua notes too, frequently the students are unable to rise above the personal to create a more professional and critical reflection on the course in their evals. And yes, gender, ethnicity/nationality, and sexuality are all in play, as are physical appearance, clothing, etc., especially for women faculty. I like your comment, perpetua, on how men with English accents are automatically perceived as authoritative. Funny, that! My guess is that Mr. Bean would probably get better teaching evals because of the American fetishization of English accents than most U.S.-born women faculty.
LikeLike
My evaluations tend to overall good but fairly lopsided. I suffer from the same problem as above: my students either really love me or despise me (I make them read, which means that my “expectations are too high”).
What is interesting to me is that I do an exercise where why students get to “gender” me to begin our discussion of gender critically. In all the classes in which I have completed this exercise, some students are always unnerved that I am a woman with a authority.
Moreover, students dislike me because I am not “motherly”, which basically means that I don’t coddle or hold their hands through every exercise. This exercise tends to be very predictive of how my evaluations go. Also, from this exercise, my students are much more comfortable with male professors who are aloof and dominant. My dialogical and conversational style makes them nervous. So, I do get nervous that my evaluations count towards whether I am a “good” teacher or not.
LikeLike
Kelly, welcome and thanks for your comment. Can you tell us more about your exercise where students “gender” you? That sounds fascinating.
LikeLike
Where to begin? Good evaluations tend to run in inverse correlation to the number of students who are in the class with a curricular bayonet in the small of their back. Courses created by departments to automatically keep their deanly “numbers” up for productivity-measurement purposes and curricularly mandated for the students can be bought at the expense of your own pedagogic reputation, no matter how “well it goes” anytime out. Peer observations with written reviews can be valuable, but sometimes they are put in place as an antidote to the problems just mentioned. In which case they are likely to be uniformly and unhelpfully positive. You have to spend years in training to assign your first C+, but its recipient can “highly agree/agree/disagree” with sentences containing words that ze could not define, merely by being there that day the instrument is administered (and assuming the tuition check did not bounce). But administrators don’t appreciate the irony of that. Nor do some of my colleagues when I opine that the only “outcome” of a course is the lived life, or that its best “assessment” is the obituary. I think it’s pretty funny, though.
LikeLike
Oh how I wish my department would serious address the “how do we know what is good teaching question.” They say they care about teaching, but they only look at student evaluations and they seem to do this in a remarkably uncritical manner (as far as I’ve ever heard or seen). I’ve heard people assume/accuse others who do at least raise this issue of being bitter about low evaluations. Because it’s not possible to have legitimate concerns that aren’t driven purely by self-preservation?? And junior faculty are expected to be above the department average – thank goodness for some unpopular senior faculty to bring the average down some. I was asked to review a syllabus for a part timer, and I did it, but also kept saying “but, really, it’s hard to tell just from the syllabus.”
I do think that peer evaluation is a good idea. The idea terrifies me (I’ve never been peer-evaluated), and I think it should be done by different people over time — individuals can have dramatically different styles, everyone can have an off day, or an off class not all reviewers are going to be humble or self-reflective, etc. This is not a perfect system, but at least then it would feel a little bit more fair/reasonable to be criticized for teaching. (I don’t see this ever being instituted in my department).
More than anything though, I’d like whatever system or combination of approaches to be put in the proper perspective. Teaching evaluations aren’t useless, but they also are far far far from perfect. As several have pointed out already, mediocre or lopsided evaluations could be fine. Or could reflect excellent teaching. Or could reflect bad teaching — hard to tell, really, just from that. I tend to be a little skeptical of uniformly stellar evals, because I just can’t imagine how one could challenge the students and be loved by all (that tells you something about my own evals — pretty decent, but always with a few haters).
I also wonder if some of this “compensating for mediocre teaching” actually reflects student dissatisfaction with institutional issues — like large lectures class. If that’s the case, and that’s reflected in lower evals, then it is entirely reasonable to cut faculty slack for something they have no control over.
LikeLike
Cultural expectations play a big role in how well students perceive they are being taught. When I teach a senior-level classes, with both undergraduates and new graduate students, there are big disparities in how I am perceived. The largely American undergraduates are more comfortable with a more easy-going professor. The foreign graduate students seem to prefer a much more authoritarian instructor, presumably modeled on their teachers overseas. (We have a lot of graduate students from places with pretty rigid academic cultures, particularly China, Japan, Iran, and Eastern Europe.)
Often times, I think I get a better picture of how I’m actually doing as a teacher from the student evaluations than from the peer reviews of teaching. Criticism from other professors tends to be that I’m not teaching a class the way they would have taught it—which is not useless information, but it’s not necessarily a problem with what I’m doing either. Students just say what they had problems with. Sometimes, I need to read between the lines a bit, and of course, sometimes what they say is plain incomprehensible, but most of it is directly useful.
LikeLike
LOAF, good points. I think perpetua’s notion that evals are good for alerting one’s colleagues to a problem is right, and that it’s the faculty’s responsibility to diagnose and suggest remedies (or eventually, to administer penalties). I think you’re right that uniformly positive evals might indicate that there’s not enough challenging going on.
In my department, junior faculty are evaluated annually by one member of the T & P committee, and it’s a different person every year so that we all get a chance to see each of the juniors in action. I think it’s a good system, but I do fear (as Indyanna suggests) that “sometimes they are put in place as an antidote to the problems just mentioned . . . . [and] are likely to be uniformly and unhelpfully positive.” Then again, people have to show some kind of effectiveness and moxie just to get a job in the first place, so maybe it’s true that we’re already Lake Woebegone-above average after all? (I think there are lots of good people who don’t get jobs, and I’ve never met an abject incompetent in a t-t job.)
I like Indyanna’s relaxed calendar for evaluating good teaching (“that the only ‘outcome’ of a course is the lived life, or that its best ‘assessment’ is the obituary”), but I recognize that the tenure and post-tenure review clocks are incompatible with it.
LikeLike
Buzz–I’m glad you find your evals helpful. I wonder if STEM students are just more self-selected than many humanities students? (That is, most of us here are talking about teaching core courses required for graduation, whereas you seem to be talking about your courses focused on your majors.)
Good points about the different styles that different students appreciate. I wonder how women faculty fare with both your U.S. students and your foreign-born students who like more authoritarian professors?
LikeLike
I’ve only been teaching for about five years, but the one thing I have consistently found to be true about student teaching evaulations is something I’ve also found to be consistently true about personal interactions — if there is a class you *hate* *hate* hate* teaching, students hate it too. If you love a class, they love it/you back (ditto individual people).
I can’t predict with exact accuracy where my evals are going to be, but I’m not surprised by them: feel dread about a class? Students don’t like it either. Feel enthusiasm? Ditto students. Now, which is the leading edge — me or them — I don’t really know; I definitely try to be enthusiastic about all of my classes from Day 1. But in some classes that enthusiasm stays up all semester long, others it flags early on — is that my fault or their fault or is something wrong with the material or are aliens sending bad vibes into the room? But anyway, I’ve never disagreed with student evals though they have made me sad sometimes: either we all were having a good time in a class or we all were having a bad one.
LikeLike
Just to add another thought to the mix about how student evaluations are often derailed by personal bias. I had a professor in college whose course a few of my other friends had taken. These friends weren’t really interested in the material and one of them, who was not a major in this particular field, withdrew from the course. The other, who was a major and is now in graduate school, stuck the course out but hated it. These two friends took the course before I took a different course with the professor and the one who stayed with it told me that the professor was racist. (These two friends and I are of the same racial minority.) The friend who dropped did so because ze wasn’t interested and didn’t want to put in the work; funny enough, ze doesn’t believe this professor is racist in the least. The friend who stayed still, to this day, believes said professor is racist, despite the fact that ze knows this prof has been my most supportive mentor.
The point of that anecdote is this: even those students who take the subject matter or the discipline seriously (even those who plan to go to grad school after) aren’t immune from reading into situations and making nonsensical judgments. (I wasn’t there for the incidents that made this student accuse said prof of racism but I do know my experience with this prof has been so far from that that I’m highly skeptical of hir claims.) It all goes back to my theory that undergrads don’t know jack. I don’t mean that in a bad way, just that they don’t really know what good teaching is (other than an entertaining lecture) because they are still learning.
LikeLike
Kathleen–welcome, and thanks for your comment.
Enthusiasm counts, I agree. But, it’s not a foolproof barometer, and it’s not entirely in our control as instructors (although it’s more in our control than anything else. A bad room can bring a class down–buzzing, bad lighting, bad drafts/heat/cold, etc.) I’ve had your experience, and then again I’ve had the experience in which I thought a class was really gelling and that we were having some great, open, life-changing conversations, and then have been slammed in the evaluations by (some of) the students. This is not necessarily proof that my perception was wrong–just that what I thought was great was enormously threatening to some of the students, and they took it out on me in the evals. (Well, I’d rather they do it there than in class on other students!)
I’ve also noticed that there are whole semesters where every faculty member I know comments that the students are really low-energy and seem depressed. I think last spring was one of them. I think campuses go through these cycles.
LikeLike
I just wanted to respond quickly to the issue of “what makes good teaching,” and to GayProf and Historiann’s comments in particular about teaching critical thinking. I completely agree that this is a very important component to what we do, and I dislike as much as everybody rote memorization of names and dates. But there is a certain amount of it that I find useful and important – that is, history isn’t just about critical thinking, but also about acquiring a certain level of knowledge about the past. Because I work on the distant past, a past that students are far less likely to know anything about (in contrast to US history for example), there is a certain amount of “landscape” I always have to provide in order to be able to move on to the critical thinking part (which I do largely through the primary source readings/ writing assignments). Many of my students are not capable of high levels of analysis and written self-expression – and while I’m not suggesting that I give up on trying to get them there, I don’t feel like a failure if all some students get out of my course is information. I find it challenging to figure out how to balance the needs of the range of students in my courses. (state school, even upper level courses have 40 students, etc.)
LikeLike
thefrogprincess–great comment. But, could it be true that this prof you refer to is in fact a racist AND your most supportive mentor? (But, I like the bigger moral of your story, which is that undergrads aren’t the most reliable assessors of teaching. Is it possible that this professor didn’t seem to appreciate your friend in the way ze thought ze should be appreciated, as a serious student in the discipline? Maybe ze was angry that your professor gave hir lower grades than ze thought ze deserved?)
They’re still learning, but aren’t we all? (I should hope, anyway?)
LikeLike
When I got senior enough to review piles of other people’s evals in consideration of hiring them with tenure, I finally observed how intensely sexist these things are. When it was just me being dissed, naturally I figured I deserved it.
Students wreath their male instructors (even very junior or non t-t ones) in the authority of the institution. If he’s tough, that’s because the discipline and its demands are tough. A tough woman is a psychobitch. If he’s gentle, that’s an act of grace and kindness calling for intense gratitude. A gentle woman must have mush for brains.
There’s a whole literature on gender bias in student evaluations, all of it vexed by the null hypothesis: what if women really are inferior? So my evidence is fragmentary. Like lunchtime faculty meetings where people have classroom gigs at 2:00–the women, even very senior and accomplished ones, leave early “to prepare,” while the men breeze right from lunch table to podium. Or talking fast: she must be nervous, but he’s a genius who has to remember we’re not all at his level.
I think my evals are pretty good, but at this point I don’t look at them. Deans read them and say positive things and I figure if there’s a problem, they’ll let me know. I could learn from the comments, but the chaff to wheat ratio is way too high, and I don’t want to get upset without a good reason.
LikeLike
In response to LadyProf and the differences in pre-class prep between male and female faculty members – when I was a young grad student, about to TA for the first time, I was at a student-staff roundtable designed to field questions for new TAs. I asked how former TAs dealt with “turning off” – how to mentally take a break from teaching to focus on their own work, and I was told by a senior faculty member that he understood whee I was coming from, because, you know, I’m a woman, and inclined to be maternal, and want to coddle, but I just needed to act like a man and put my own work first. It is perhaps, no wonder that some students also buy into these views.
LikeLike
Historiann, thanks for the welcome, I was really glad to come across your blog!
I don’t always read the comments on student evals, maybe this is terrible. But I feel the way Ladyprof does: I’m interested in the overall numbers, and the rollercoaster of anonymous comments (the gamut from “this prof is the greatest” to “I hope she dies so I can be first in line to pee on her grave” — okay, not direct quotes, but you get the idea) is not good for my inner equilibrium. I’m also not sure they are as helpful as averages are, at least in classes of 20+ students. Individual students aren’t experts on pedagogy, but an aggregate group probably can indicate whether “stuff was learned” and “we enjoyed learning it”.
I don’t think students can tell you how to fix a class; they can tell you if it’s generally working or not.
LikeLike
In my department junior faculty are required to get peer review of their teaching *once* before tenure. This means in practice a tenured member of the faculty in your area sitting in on a single lecture and looking at the class design (syllabus, assignments, etc.) There is not, however, any official push to make this happen early. So in my case I was observed in the very last lecture I gave before coming up for tenure. It was a dispiriting performance that made no one proud. The biggest criticism I got was actually that I could show more of a sense of humor to engage the students. I got confirmation that my colleague thinks I have a good sense of humor (I do!), but I didn’t think the Civil War was a good topic for laughter.
For me any sensible peer evaluation should actually include a different form of peer mentoring–I would like to be able to observe my senior colleagues’ lectures to see what different models of successful teaching looks like. This is a touchy topic for some of my colleagues. One in particular has a huge investment in the idea that his way of teaching–he gives fully written out prepared lectures in the old-school way–is the only model of good teaching, and he takes any suggestion that other ways of teaching have merit as an attack on his style. There are a couple of other people in my department who have the same general approach. By and large few people like being observed and compared to others. But I wish that had been an option. I think that you ultimately learn by doing, but modeling is an important way of learning (and teaching).
I should also say that I have found designing and teaching graduate seminars to be more demanding than I anticipated. (I teach in a dept where there’s usually 10-15 Ph.D. students a year, and no masters students.) It’s not that I thought it would be *easy* per se, though I did anticipate that grad students would need less prodding to talk. In my previous experience, they loved to talk, and talk, and talk. But when I thought of designing grad seminars as exercises in steps in the professionalization process more than just “classes,” I found it more difficult. I guess I don’t want my students posting to this blog complaining they have been raised by wolves in the wilds of Norcal!
LikeLike
At Colgate, I taught the same interdisciplinary gen ed class both fall and spring. The Fall section was a mess in places – I’ve since referred to it as the rough draft of the course. In the spring, it was firing on all cylinders, and even from the students’ side the grades were embarrassingly high. However, the fall evaluations were noticeably better, with about a 12-1 positive/negative ratio rather than 4-1, even though it was bolstered by people who took the course on the recommendation from Fall students and who therefore knew what they were getting into. The entire difference was in the students’ level of interest in the material, and that may in turn have been caused by a time change which led to more students from the hard sciences who are less likely to care about non-science gen ed requirements – the negative reviews tended to say things like how they took this class only because they had to, in one case attacking the concept of the liberal arts in the process.
LikeLike
My point in the above post was how students evals are imperfect indicators – there is absolutely no way my fall version was actually better than my spring.
LikeLike
John, that’s derelict peer evaluation of your teaching. I like your idea–that junior faculty should be permitted to “review”/learn from/be reassured by the mediocre teaching of their elders. It sometimes seems like teaching is like sex–we all think about it and even talk about it a lot, but no one wants to let anyone else watch them do it.
I’m with LadyProf and Kathleen on the value of specific comments after your first 3-4 years of teaching, or after switching institutions. I’m also totally with LadyProf on the sexism of the instruments. I also think that (as other commenters have suggested) that anyone who isn’t a white man with an American (or English) accent is prone to unfair evaluations of their performances. I haven’t seen overtly racist language used in evals of nonwhite faculty members, but there is a tone of pervasive doubt (as in the evals of women faculty) about the faculty member’s expertise, and/or accusations that the faculty member isn’t “available” or “accessible” enough. (Also a charge levelled against women faculty who don’t play the mommy sufficiently, as someone else upthread suggested.)
A colleague of mine from another institution had a great idea, one that I should start using again: She gave them instructions on how to write an effective student evaluation. Now, that may sound like she was gaming the system, but in fact I think it got her a better *quality* of student evals, as well as somewhat better evals overall.
In short, she spent 10 minutes talking to them about how if they’re entering the professional/managerial class, they’ll be expected to evaluate the work performance of others, so they should use this as an opportunity for professional development rather than as a chance to get back at a professor who’s been annoying them all semester. She pointed out that personal insults, comments about the professor’s appearance, and other ad hominem attacks impeach the overall value of the instrument, so she urged them to think about how their evaluations reflected on them as evaluators. She also taught them how to write a “praise sandwich,” as in, “I liked W and X/Y could have been better/but I liked Z too,” because if you can show that you see value in the job as it was performed, it makes you sound like a more credible evaluator.
LikeLike
One other wrinkle I should add: Colgate doesn’t use any statistics on their evaluations – it’s all long-form answers. Opinions vary, but this both prevents administrators from using the statistics as a crutch and allows a more nuanced sense of where opinions are coming from.
LikeLike
Brian: funny story! I’ve had similar experiences. Alas, 19-year olds don’t in fact know everything. It sounds like you were perhaps in part a victim of your fall term’s success, as well as guilty of making them take more responsibility for their learning.
Your story underscores the importance of peer evaluations. You could certainly explain to a colleague why your spring course was in fact better than your fall course by showing them your syllabi, and explaining why you made the changes you did and exactly what they were. Your students probably would be focuse on “well, they only had to write 3 papers and we had to write, like, 5!”
LikeLike
Actually, most changes were organizational. The topic was the Middle East, with the point being to give students the chance to explore a non-Western culture and it’s relationship with the West. All assignments were the same structure, but I’d learned what topics were rushed, which were drawn out, what readings wouldn’t work, and that sort of thing.
A good example is the two weeks I spent on Middle Eastern history from 600-present. Students, of course, squat. In the fall, I tried to get them to pull key themes out of readings, which was a disaster. In the spring, I did more spoonfeeding those two weeks, which were mostly lecture (complete with Roman numeral outline!) and discussions over what I declared key themes – poring over texts came later, once they’d established some background. In the second semester, however, a couple of people complained that there was “too much history,” or that lectures were confusing. (Students are usually more forgiving of flawed discussions, at least in my experience.)
Two other things I’ve noticed:
What you do in the final month counts for far more than what you do in the first month.
If students complain about something in class, they’re less likely to write it on the evals. This may, of course, be because of self-correction.
LikeLike
At my place you’re subject to annual reappointment each year for the first five years, before proceeding to tenure review, and all of this is elaborately metricized. You’re “observed” (and written up) by two colleagues each semester, plus the chair once a year. So you get thoroughly used to “doing it in the road,” per the Lennon/McCartney metaphor and Historiann’s reference. But as I suggested above, this process is largely designed to build an affirmative record that would offset any hypothetical damage from evaluations in the “bayonet” survey class that all students have to take, and many learn to hate. So it probably all comes out as the wash in terms of credible feedback.
The other thing is, per regulation, evaluations have to be physically administered by someone other than the instructor. So, unlike the place where I visited for three years–where you bring them in, designate the kid in the left front row to collect them, emphasize that you won’t see them for weeks, and walk out–the entire faculty has to play musical chairs with each other. This becomes a nightmare organizational task and piece of committeework for a colleague, and it requires the evaluations to take place over the last three weeks of the semester, rather than at a late meeting where a substantial part of the class happens to show up. Not at all clear how any of the resulting signification gets to be called “data.”
LikeLike
Just to respond to what Historiann’s response to my earlier comment: it’s certainly true that the professor’s exceptional mentorship doesn’t preclude hir from being a racist. But ze was described to me as a racist before I’d ever taken a class with hir, so I was on the lookout. Generally, you find what you’re looking for if it’s there to be found. I suspect that my friend had a personality clash with this professor (yet another way evals are skewed) that was exacerbated by my friend’s not getting the grade ze thought ze deserved, as you suggested.
They’re still learning, but aren’t we all? (I should hope, anyway?)
Absolutely. I think student evals can be quite useful to the individual instructor. But they’re often too skewed an instrument to beat the heads of professors with.
LikeLike
Brian, I’ve been in a system of all narrative evals, and got completely screwed because they were summarized (no one will read them all!) and done negatively. So I mistrust all narrative just as much as numerical evals.
Ann, I love the idea of teaching students to write evaluations.
LikeLike
re: gender bias. I am absolutely convinced that gender bias is pervasive in student views of faculty, and hence in faculty evaluations. I’ve never seen a department that acknowledges this or takes it into account in any fashion. But along with it of course is racism, which I’m sure follows the same lines. And not to belabor the accent point, but I’ve seen many talented academics/ teachers given withering evaluations because the students ‘couldn’t understand hir. [ie teacher in question is from Africa and *perfectly* comprehensible.]” It’s a real problem that the academia takes evaluations into account for tenure, but not these other factors.
(I’d love to do a study that cross-checks the number of male teachers with British accents to their evaluations!)
LikeLike
There is one other factor that I certainly see in my teaching evaluations in lower division courses: the quality and/or likability of the TAs. Students fill out separate eval forms for TAs and professors, but a better (or better liked) TA always raises scores. Sometimes this comes through overtly–the written comments on the course will include a “Ben was awesome as a TA!” thrown in there. But since students all have to put the Section # on the Scantron form, I can spot more general trends as well.
There’s also another issue wrapped up here that I am curious about, and that’s grading. I have had very few useful discussions with my colleagues about 1) how they grade assignments or 2) whether or not they adhere to any kind of vaguely defined “curving” or “norming” in class. How do you evaluate each essay? How do you determine if grades in a class are too high or too low?
(Full disclosure: I don’t create a curve for the class, but I do try to hit a “sweet spot” for every test and assignment where the mean and median scores are somewhere between a B- and a B. In other words, if the average score on the midterm was a 90% it was too easy; if it was a 75% the test was too hard.)
Learning how to grade–and specifically learning how to grade in your department, at your institution–has significant pedagogical and professional repercussions before tenure. (It’s the most commonly mentioned item on my student evals, for example.) But I think lots of teachers just seem to assume that we should just intuitively “know” what makes an A, or a B, etc., regardless of whether you’re teaching a lower division course, an upper division one, or a seminar. Moreover, most people in my department want to avoid being seen as the person who is too hard or too soft on the students, so I have found many somewhat unwilling to engage the subject in depth.
LikeLike
Susan raises a good point–not having numbers doesn’t mean that the “data” (pace Indyanna’s caveats) can’t be massaged.
perpetua–your accent study would be hillarious!
LikeLike
I constantly fret about the quality of my teaching, even as “just” a one-night-a-week adjunct. My U offers teacher training, but it’s always during the day. It’s a tough juggling act to be engaged, teach the subject matter, and (hopefully) get students to start questioning things, all in a long class once a week. I’m currently teaching the drive-by survey, Intro to Physical Anthropology, where each chapter of the text contains enough material to fuel several life-long careers.
I agree with perpetua, “Many of my students are not capable of high levels of analysis and written self-expression – and while I’m not suggesting that I give up on trying to get them there, I don’t feel like a failure if all some students get out of my course is information.” If most of my students get out of my class knowing the basics of natural selection and that, not only are we not descended from chimpanzees, but that we never lived with dinosaurs, I’m good. If some come out of my class with bubbles popped, widened horizons, and a head full of questions, I’m very happy. The best teacher feedback I’ve received? A grade school/high school student I mentored went on to study Anthropology as a grad student (and I’m thrilled to have her as a colleague, even though her specialty is a world away from mind) and a student I had 3 years ago took another course from me this past summer for the hell of it.
Departmental reviews? Last time I was peer-reviewed in the classroom was 4 years ago. As intimidating as it was, I kinda wish they’d come back!
LikeLike
PS: John, “I would like to be able to observe my senior colleagues’ lectures to see what different models of successful teaching looks like.” Me too. I want to see the kickass teachers and the ones that …uh… not so much. That way, I can learn new things, and feel better about where I’m at.
LikeLike
Re: Student Evals. One year, due to a TA strike and management’s insistence we get grades in on time, I offered students in my 300-person survey the option of not taking the final exam. 99% took that option (thankfully!).
My teaching evaluations were much higher that semester than any of the 6 other times I’d taught the identical course. My lesson: give them less work (and high grades) and you’ll get great evals.
Why in the world do we let unknowledgeable 19 year olds be the primary means of evaluating our teaching skills? Probably because at many institutions, we don’t take teaching seriously — lord knows we rarely explicitly learn about it in grad school. A colleague purposely invites his smaller classes to his house for dinner — and hands out the evals after dinner. Who is going to write nasty things after being fed and invited into someone’s home?
If Unis were serious about evaluating teaching, we’d have professionals do the evaluation in addition to discipline colleagues.
LikeLike
A couple of thoughts:
1) commenting on early in the thread about assessment in the obituary – I wonder if giving a graduating senior a list of all their classes and asking them to note something they learned from the class would help give a time lapse view of the value to students – or not. Some kids nurture grudges.
2) I use my own undergrad experiences as models for teaching and I had some terrific teachers. I’m not saying I’ve managed to reach their level but I have goals without necessarily observing my colleagues.
3) My system that I use sometimes and not others times – I give students a document or a film clip (something to analyze) that is relevant to the class and a question to answer on the first day of class. I give them the same exercise in the second to last week of the class. I then make photocopies for myself and return the two copies to the students in the last week, around when I would be administering the evaluations (if, of course, my school hadn’t switched to lame online evals!) Then students can see their answers at the beginning of the course and at the end and they have something slightly measurable to consider if they have learned something in the class. It also allows me to get a sense of what they might have learned in terms of analysis in the class.
4) I’ve quit looking beyond the numbers for the most point on my evals (discussion at the end of the comment thread). All through grad school I received great evals. But at my first t-t job I received HORRIBLE evals. I was utterly demoralized. I immediately moved to a different job – and then again received great evals (I’m still in touch with several students from that school who have now graduated) and eventually moved again – and had fine evals at the school I now work at. I’m not rocking anyone’s world but I hit at least the average in my dept and above average in my college. That is all I’m aiming for. The depression that came with the one school taught me to look at the numbers. I generally can tell and have an open enough dialogue with students during the semester to know what to fix and what is working.
LikeLike
I just wanted to second Buzz’s comment that peer review from other profs can be about you not teaching the class the way they would have taught the class. I mean, most of us didn’t receive a lot/any training in pedagogy; neither did our colleagues. In my own case, such reviews probably indicated a poor fit between my teaching style and the culture of the institution, so even if they didn’t reflect my teaching in an absolute sense, they were a decent metric for the success of my teaching in that particular context. I have a friend, though, who is an amazing teacher – he pushes students to work hard, to make connections between the work on their lives (he teaches 20th c.), not to make assumptions, to analyze sources, how to research, etc. All sorts of things you’d want a teacher to do. But because he is an intensely casual, relaxed person, who will on occasion swear in the classroom, and who, when (say) teaching the progressive movement, will make fun of students’ drinking habits, and generally doesn’t present himself as The Authority, one of the other members of the department hates his teaching. Just HATES it. And this person has been in charge of evaluating this friend of mine year to year. My friend is insulated by spectacular teaching evaluations, but it’s still difficult for him at times.
That said, peer evals can definitely be helpful, and avoid some of the worst problems of student evals. They’re just not a panacea, sadly.
LikeLike
Thanks for the welcome. I love this blog for cutting edge feminist analysis and snark.
I think I might have missed the moment, but here’s a summary of my gender exercise (which I borrow from a fabulous mentor).
The exercise is not for the faint of heart because I basically stand in front of my classroom and tell them to critique my gender performance. So, it usually begins with commentary on my dress and style, hair, and of course, my shoes, but my students quickly begin to deal with other things like body language, voice and attitude. What is really fascinating is their impressions of things that our less tangible like my overall attitude and disposition. I generally exaggerate my dress to be more feminine (pink is included) to make it easy to start.
What I really like about this activity is how quickly they realize the ambiguity of gender from just my example, and I then have them reflect on their own gendered performances. Usually, my evaluations have a couple of comments about the exercise and how “good” my gender performance is.
As said earlier, I usually find comments from the exercise and evaluations match up. Some students are really irritated that I am not more feminine, which to them means motherly, and that always appears in the comments section. Feminine dress, I guess, doesn’t coincide with tough grader.
Oh, and when I wear glasses instead of my norm of contacts, some students think I am smarter and more masculine. Strange.
LikeLike
Great comments, everyone. New Kid, I think you’re entirely right. I think departments should think about evaluating teaching the way they think about evaluating students’ progress and performance in a class. That is, few of us assign ONLY papers, or ONLY multiple-choice quizzes, or ONLY essay exams (to name just a few instruments for testing students). So taking teaching evaluations into context along with reviews of syllabi and teaching materials, peer evaluations, and other materials (thank-you notes from students, etc.) seems to be the only fair way to go. That way, the biases of each kind of evaluation are mitigated as much as they can be by other evidence.
Liz2’s idea for showing students what they’ve learned is brilliant. I just might borrow that, along with Kelly’s “gender assessment” exercise. (On the glasses v. contacts thing, except to say that it plays to the old, old Hollywood trope of using glasses to “disguise” the extremely glamourous secretary played by Marilyn Monroe or someone like that. Glasses are also a proxy for intelligence, and we all know that beautiful women can’t possibly be intelligent!)
Shaz’s experience in getting the best evals of her career in a semester in which she made the final exam optional: AWESOME! I would looooooove to try that, too. But this is my suspicion, always: students give high marks to gut classes. Classes that challenge them aren’t appreciated as much, certainly not in the moment.
LikeLike
BTW, many of you have given me some ideas for a post on “what exactly is good teaching and how do we know it when we see it?” I hope most of you will come back for the discussion of that one, too.
LikeLike
To a certain degree, I think really brilliant teaching is like brilliant dancing — it’s gorgeous, and rare, and everyone knows it instantly when they are in its presence. The profs who have students clambering over one another to enroll in their classes, win tons of awards, get stellar evals year after year — they are clearly great. But they probably constitute 5% of the professorial population, and always will.
Beyond that, the dirty secret of teaching is probably that it has a pretty bell-curve like distribution: there are a few great profs and a few awful ones, and then a huge bump in the middle of most of us who try hard and are okay and are viewed with enthusiasm by some students and disdain by others; we connect in some classes, and with some students and our colleagues connect in other classes with other students.
All the emphasis on “excellent teaching” asks the impossible; what I think would be more useful would be for universities to think about how to keep students and profs alike from becoming cynical about teaching and learning.
If you are encouraged in the expectation that all your profs should be GREAT teachers, you are going to find most of your profs pretty thin gruel. If you are uniformly exhorted to be GREAT teachers, you are going to find student evaluations threatening and hurtful.
What if universities said, hey kids, here are the great profs: X,Y,Z. Try to take a class from one of them at some point, you’ll find it a real treat. Our other profs try hard, too. If they are awful, we don’t keep them on. But mostly, you are going to have to meet them halfway — they are really trying — and you probably are even going to dislike a few of them from whom you nevertheless have to take a few classes.
And, you know, universities could say to profs — we know you are making the effort to view your students with the same reasonable, generous attitude with which we encourage them to view you. How can we promote that as the basis of the best possible teaching and learning environment given the irregular mass of humanity that makes up this (and any) uni?
LikeLike
Well, that’s just it, Kathleen Lowery–our malarkey about “excellence.” Teaching is a quotidian task inside a university. The idea that teaching isn’t good enough unless it’s upper-elite is madness … but such is the mindset of people who came of age when their own little academic excellence won them formative praise.
We treat our students the same way.
One can debate which reform is least doomed. I personally despair of eliminating elitism in higher education, and would instead focus on treating workers fairly. Criteria used to judge our work performance, including our teaching, should be fair. No moving the goalposts; no defining Excellence as the characteristics of people already favored.
LikeLike
Kathleen, sorry for misspelling Lowrey. I hate when people misspell my own name and should have been more careful. “Malarkey,” a word I hardly ever use, was the culprit, the anticipation of which addled my fingers.
LikeLike
I completely agree with you both, Kathleen & LadyProf. That’s how I felt when I read Historiann’s original remark about what’s wrong with a little mediocrity. That’s a negative word in our culture, of course, but the bell curve analogy explains the situation better. We’re always trying to be GREAT when the reality is so much more mixed. Why isn’t ‘mixed’ ok? Mixed should be perfectly fine, since our classes our mixed, and most of us are faced the Herculean task of instructing audiences of wildly different skills – from the functionally illiterate to brilliant. And regardless of skills, every student has different learning styles and preferences.
There is one other big issue we’ve talked too much about regarding evals:the fact that many/some of the highest evaluated teachers are also the higher graders. The inevitable sentence for them to rate the professor’s “clarity of grading expectations” or whatever they say, it obviously all about “did you get the great you think you deserved?” Any professor who stringently rejects inflation will have low(er) evals.
I think I’ll start wearing my glasses to class. It was really hard for me at first because I was a young assistant prof – well really not that young (30), but I looked even younger, so I made up for it by being a complete hardass. That worked pretty well, actually, though it did scare the pants off of some of them. (I have so far escaped comments about “motherliness” but now that I am a mother, I know they are a-comin’.) I had a student who was so terrified of my “draconian” late policies that he sent his dad to class to explain his absence – he was having emergency surgery on his appendix!
LikeLike
Pingback: What is good teaching, and how can we know it? : Historiann : History and sexual politics, 1492 to the present
Kathleen, LadyProf, and perpetua–I think you’ve got it exactly right why I think “excellence” is such a vague and unsatisfying goal, and why “mediocrity” is OK. Who really can be “excelent,” given the ways in which our identities powerfully shape our students’ reactions to us and to our classes? My guess is only donnish, middle-aged tall white men with English accents.
I too am concerned by the ways in which grade inflation plays in student evaluations. I’ve heard that there was a study a decade ago (or more) that showed that the evals students gave their teachers correlated almost exactly with the grade they thought they were getting in the class. (So you give me an “A” and I’ll give you an “A,” and so on.) I don’t play that game. I think students need to share some of the risk for being in my classes, and to take responsibility for their own learning. That in the end is more important to me than my student eval numbers. (Maybe easy to say, because I’m tenured now! But it was true even when I wasn’t, probably because I’ve landed in a department where student evals are regarded skeptically and are only one part of the evaluation puzzle.)
LikeLike
LadyProf — no problemo, but thanks for noticing! Since my last name has at least 4 different variants, it happens all the time but like you I do always give a slight twitch 🙂
Perpetua — good luck with the glasses; when I was interviewing for tt jobs in my early 30s I actually bought fake eyeglasses halfway through the process for one campus interview! I didn’t get that job either, and after that went back to au naturel (and eventually landed a job). But anyway, I do think teaching is 50% performance and props can help, the bigger a class the more formally I dress for it, for ex.
Historiann — I’m really looking forward to the next round of conversation!
LikeLike
Ease of grading doesn’t always correlate with higher evals. In my case, I always have been proud of the fact that my uni evals., as well as my Rateyourprof. comments, very frequently mention that I am a tough grader… yet I get very high scores.
I think I know exactly the reason why, though: I’m super easy about deadlines. I tell them when I hand out assignments that I will readily negotiate with them about extensions — no questions asked about why they need the extra time. (In huge classes like the ones I teach, it really makes no difference in grading turnaround to get some of the assignments late, since it takes so long to go through the pile anyway). However, I also emphasize that the open extension policy is my way of being flexible, and of allowing them the best possible circumstances in which to do good work. If they still cannot generate good papers, then they will certainly get a low grade, and cannot complain of not having had every consideration. This system, and my frank explanation of it, strikes them as eminently fair, and they often give me high evals. while simultaneously noting that I am a hard-ass grader.
Also, I think I come across like New Kid’s friend: very casual, open, and somewhat eccentric. The kids at Office Park U seem to like that, too.
LikeLike
Pingback: Worthwhile blog reading | Geeky Mom