They’re almost universally loathed by professors as being too subjective and an unreliable indicator of performance. But beyond that, surprisingly little is known about student evaluations of faculty teaching. How many colleges require them, and what do they ask? How many students complete them, and what effect do they have on instructors’ careers? A committee of the American Association of University Professors wanted to help answer some of the questions, and help stir discussions about a better way to rate professors in the classroom. Survey responses gathered by the committee from some 9,000 professors suggest diminishing student response rates for course evaluations, too much focus on such evaluations alone in personnel decisions -- especially for non-tenure-track faculty -- and a creep of the kinds of personal comments seen on teacher rating websites into formal evaluations.
But while the committee argues that whatever value student evaluations ever had is shrinking, it says student surveys can play an important role in a more holistic faculty evaluation system. “I’m a department chair myself, and it matters to me to get some feedback from students about how their experience in the classroom was,” said Craig Vasey, who heads both the AAUP committee that conducted the study and the department of classics, philosophy and religion at the University of Mary Washington. “But [student evaluations] have to be supplemented by class visits by peers and reviews of syllabi, and participation in ongoing faculty development.”
Noting that one survey respondent had offered up what is a perhaps a more fitting name for student evaluations -- “student satisfaction surveys” -- Vasey added, “We’re not calling for them to be abolished, but there’s something dishonest about what they are and how they’re being used.”
Last fall, the AAUP’s Committee on Teaching, Research and Publication sent out 40,000 invitations to tenure-track and non-tenure-track faculty members to participate in its online survey about teaching evaluations. It asked questions about institution type and required mechanisms for evaluating teaching -- such as student evaluations on paper or online, development of teaching portfolios, engagement with an on-campus center for teaching, and evaluation by peers or administrators. It asked about the existence of a faculty mentoring program, how student teaching evaluations are crafted and by whom, and faculty members’ feelings about them.
The committee received about 9,000 responses back. The majority came from tenured professors (54 percent). Some 18 percent came from full-time, non-tenure-track faculty members and 15 percent from tenure-track, not tenured faculty members, according to a write-up of the data to be presented at the annual meeting of the AAUP in Washington this week. Most respondents were from four-year, teaching-intensive institutions (48 percent), followed by four-year research institutions (35 percent). The rest were from community colleges or professional schools.
Here’s what respondents said: frequent use of online and paper evaluations is now about even. Required use of quantitative evaluations beat required qualitative evaluations, at 55 percent versus 44 percent, respectively.
Respondents who said their institutions had adopted online evaluations reported much lower student return rates than those who stuck with paper evaluations: 20-40 percent versus 80 percent or higher.
“With such a rate of return, all pretensions to ‘validity’ are rendered dubious,” the paper says. “Faculty report that the comments coming in are from the students on either of the extremes: those very happy with their experience and/or their grade, and those very unhappy.”
Faculty members said they had little to no input in crafting evaluation instruments, and pointed out that teaching in one field is quite different from the next -- something evaluations should reflect.
In a comment section the survey, some faculty members said they’d seen the kind of “abusive, bullying effects of anonymity that are today pervasive on websites… making their way into student evaluations,” the committee says. “Women faculty and faculty of color report negative comments on their appearance and qualifications, and it appears that anonymity may encourage these irrelevant and inappropriate comments and attacks, which are sometimes overtly discriminatory.” Those findings are in line with recent research suggesting strong gender bias in student evaluations.
Other professors talked about how being a tough professor works against them in student evaluations. Here’s an example: “My students often give me (I'm a woman) lower course evals than my peers because I assign a lot of work and hold them to high standards. They don't like this in the moment, but I know from talking with them that a few years later, students are able to see the ways in which this work influences their current abilities and vision and they are grateful. But I don't get the benefit of this perspective.”
Most evaluations are done in the last weeks of the semester, according to the survey. Some institutions allow students to complete the evaluation even after they’ve received their final grade, potentially compromising objectivity.
Some 25 percent of professors say their evaluations were frequently published for students and others to see. Other means of evaluation vary. About half of respondents said they were evaluated frequently by administrators, and about two-thirds by peers.
“The development of teaching portfolios, mentoring of junior colleagues or teaching assistants, or engagement with an on-campus center for teaching and learning, while often recommended, was tagged as required by very few respondents,” the committee says.
Roughly half of respondents reported a mentoring program for junior faculty on their campuses, but few were involved with one. And while 75 percent of respondents said there’s a center for teaching and learning, most said the centers were better known for helping instructors with technological needs than pedagogical ones.
Most agreed that teaching and learning centers demonstrate a campus’s commitment to pedagogical excellence, and 86 percent supported the idea of mentoring programs for junior faculty. Even more respondents (90 percent) said institutions should evaluate teaching with the same seriousness as research and scholarship. While two-thirds of respondents said student evaluations create upward pressure on grades, some 77 percent were against the idea of quotas to fight grade inflation being imposed by the administration.
Who decides what goes into a student evaluation instrument? Some 55 percent of respondents said that was not the job of the faculty primarily. Some 62 percent said decisions concerning the use of student evaluations in personnel decisions, such as promotion, tenure and merit, did not lie with the faculty.
Over all, some 69 percent of respondents said they saw some or a strong need for student feedback on their teaching. But only 47 percent said teaching evaluations were effective. “We saw numerous claims that faculty are evaluated and recommended (or not) for contract renewal or promotion as a result of the grades they assigned, especially claims that there is administrative pressure to pass many students who deserve to fail courses,” the committee says.
The committee pays significant attention in its write-up to adjunct faculty concerns, to include graduate teaching assistants, saying that most respondents noted that traditional monitoring of teaching was limited to those on the tenure track.
For adjunct faculty, the committee says, there is “significantly less support and, oftentimes, exclusion from participation in mentoring, teaching programs, instructional development and peer evaluations. Given the reality that [non-tenure-track] faculty are responsible for teaching the majority of courses and that graduate students represent the next generation in higher education, this lack of mentoring and attention to quality seems disturbing and a cause for concern.”
The committee also argues that online course evaluations, with their low rates of return, “aren’t working” for any faculty member, tenure track or not. It endorses having faculty within departments and colleges -- not administrators -- develop their own, more holistic teaching evaluations, and they raise the possibility of ending student anonymity, saying that students might be more accurate and fair if required to give their names.
Perhaps most importantly, the committee calls on “chairs, deans, provosts and institutions to end the practice of allowing numerical rankings from student evaluations to serve as the only or the primary indicator of teaching quality, or to be interpreted as expressing the quality of the faculty member’s job performance.” Addressing adjunct faculty concerns, the committee adds, “We especially call on administrations to stop the lazy practice of making contract renewals on the basis of such partial, biased and unreliable data.”
Philip B. Stark, a professor of economics at the University of California at Berkeley and co-author of a widely read 2014 paper that was critical of student evaluations of teaching, said he was even more against them now, given the growing body of evidence of their unreliability -- especially concerning gender bias.
“I no longer think [student evaluations] should be used in any formal way by any institution, especially not as a measure of teaching quality and especially not for the purposes of hiring, merit evaluations, firing, tenure, et cetera,” Stark said. “They do not measure what they purport to measure.”
Stark said he thought that basic items such as “Could you hear the instructor from the back of the room?” or “Could you read the instructor's handwriting?” or even “Did you enjoy the class?” might be worth collecting, but only for the instructor’s eyes.
Vasey’s committee doesn’t claim that its sample is representative as a whole. In fact, the paper discusses at length the fact that non-tenure-track faculty, for example -- a minority of respondents -- actually make up the majority of the teaching force. But it says the survey results offer a valuable snapshot nonetheless.
Adrianna Kezar, director of the Delphi Project on the Changing Faculty and Student Success at the University of Southern California, has studied student evaluations extensively, along with how adjunct faculty employment impacts student learning. She said student evaluations are the primary means for evaluating non-tenure-track faculty, and often for non-rehiring. Like the AAUP committee, she said student evaluations shouldn’t be abolished, but that non-tenure-track faculty need more robust, complete measures of performance.
“Research demonstrates that student evaluations can be valuable among several sources of input on faculty teaching but need to be combined with other sources including peer observations, syllabus review, portfolio analysis and teaching philosophy and reflection, among other approaches,” she said. “Single metrics of teaching have not been found to provide a complete enough picture for improvement.”
Kezar said via email that the issue has implications for student success, namely that overreliance on teaching evaluations for adjunct faculty ratings has led to “integrating effective educational practices such as active or collaborative learning because students often resist new evidence-based teaching approaches that require greater engagement and challenge and therefore penalize instructors who use such approaches. Faculty are often given higher evaluations if they do not challenge students to work hard.”
Of course, not all course evaluations are created equal. Ken Ryalls is president of IDEA, which offers colleges and universities research-based course evaluation systems that can control for class size, student motivation and other factors. He said he understands faculty concerns about low response rates baring statistically insignificant data, but that the correlation between response rates and teacher ratings is actually quite low.
Ryalls said IDEA has an 81 percent response rate on paper and 66 percent online, and that a mobile device delivery system eliminates the paper advantage, since students can fill it out at their fingertips.
“Even without a mobile option, response rates can be just as high online as with paper if teachers take certain actions,” he said. “Faculty can clearly communicate their expectations for student compliance, ensure confidentiality, monitor response rates, send reminders, and create a culture that values student feedback.” The most effective ways to get students to complete course evaluations is to assure them that their responses will be valued and make a difference in the future of the course, Ryalls added.
Qualitative comments, just like numerical scores, “should be used as part of a global picture and analyzed over time to look for informative patterns of feedback that may help instructors improve,” he said. Instructors also may guide students on how to write helpful comments before the surveys are given.
Ryalls said he agreed with the AAUP paper in that faculty members should have input in what questions are asked. He also supported a more comprehensive faculty ratings system. Ideally, he said, student evaluations should count for no more than 30 to 50 percent of the overall assessment of one's teaching, and ratings should be collected from at least six classes before summative decisions about effectiveness are made.
“When we stop thinking of evaluation as an event that occurs at the end of the semester and start thinking of it as an ongoing process that is based on multiple sources of information," Ryalls said, "we will begin to accept the value of student ratings gathered from a reliable and valid system.”
The World Innovation Summit for Education (WISE), an initiative of Qatar Foundation for Education, Science and Community Development (QF), has entered into a number of new partnerships and initiatives with key Chinese educational institutions following substantive discussions with a range of senior Chinese educators and research officials in Beijing. In addition to expanding Chinese representation at the WISE 2015 Summit, WISE will engage with Chinese institutions on collaborative research efforts in entrepreneurship education, twenty-first century skills and other areas of common interest.
In the education sector, there is a growing awareness of the shortcomings of traditional degree programs, and the need for more technical and vocational training which will produce graduates who have the skills needed in specific industries. In the GCC in particular, with governments looking to increase employment of nationals, there is a growing focus on developing vocational and technical training that will meet the needs of national industries at the same time as providing rewarding and productive career opportunities for citizens.
Read more: Aligning education with the needs of business
Seventy-four years ago, Henry Luce published "The American Century," an essay that argued that American culture would play a starring role in creating a global environment in which the United States could thrive. Chief among his examples was American language itself; not just English, but an American-inflected argot that would be carried around the world via music, movies, comics, and popular culture. This was for Luce the sign of an internationalism that Americans themselves hadn’t yet acknowledged.
Today few would doubt that the reach and power of American culture is global, nor that the country is an international power. Colleges take a significantly different approach to teaching about the world than they did in 1941, and American studies has sought to be more global in its outlook. More foreign languages are taught than in Luce’s time, and study abroad has become a rite of passage for many students.
Yet a creeping monolingualism is overtaking higher education, despite the efforts of so many in the trenches. The signs are everywhere: Major universities are closing German departments and cutting Russian and French programs; general foreign-language requirements are easing up. Over all, college language enrollments tumbled 6.7 percent between 2009 and 2013, according to the Modern Language Association. Despite the growth of study abroad, it is increasingly easy for college students to take their courses in English in such countries as Jordan, the Czech Republic, France, and Turkey. The widespread sense that English has become a global lingua franca contributes to an unfortunate sense that learning other languages doesn’t matter.
Arabic is one of the languages that suffers in this climate, both because of its difficulty and the resistance of many language programs to embrace its spoken colloquial forms. Although it has been the fastest-growing language of study since 2001, enrollments fell 7.5 percent between 2009 and 2013. Given the enormous military and political focus on the Middle East, it is urgent that Americans learn Arabic. If the United States is going to try to understand, rather than bomb, invade, and occupy part of the world that has been our government’s central obsession for almost a decade and a half, then more colleges need to teach Arabic and do so in a vibrant way. Higher education has never had a more crucial role to play in achieving peace.
Arabic is the fifth most common native language in the world, with at least 295 million native speakers. And it is spoken in 60 countries, a number second only to English. That means there are jobs out there for those fluent in Arabic, a multitude of opportunities in both the private and public sector, including prospects we have not imagined. But this is not the only reason — or even the primary one — to support the study of Arabic.
Studying Arabic is a moral good and a matter of our national interest. Training a new generation to understand and converse in Arabic may help to reverse the previous generation’s misapprehension of the Arab world, especially as hate crimes against Muslims continue and anxieties about the Arab world fuel misunderstanding.
I support the teaching and learning of all world languages, and I direct a program that requires our majors to study at least three years of any of four languages of the Middle East: Arabic, Hebrew, Persian, and Turkish. But I focus on the importance of Arabic here because it is the most spoken in the region — the official or co-official language of 24 nations — and U.S. college enrollments are vastly out of sync with its popularity and importance in the world.
It is also one of most difficult languages, with many national dialects as well as a formal level that is complex and grammatically rich. Part of what’s holding back Arabic study in the United States is a resistance to embracing the relationship between these various forms.
Arabic is classified as a Category IV language by the State Department (up there with Mandarin, Japanese, and Korean), the highest level of difficulty. This refers mostly to the complexity of the grammar of modern standard Arabic. So Arabic is a four-year language, in college terms, in the sense that it takes years longer to get somewhere with modern standard Arabic than it does with French or Spanish. It takes persistence and dedication for American students to make progress.
What’s more, learning the formal modern standard Arabic does little to help American college students speak the national dialects, each of them relatively distinct colloquial forms. Think of the difference between Portuguese and Italian, or Spanish and French, and you have an approximation of the difference between Moroccan and Lebanese Arabic or Egyptian and Iraqi. No one in the Arab world goes around speaking modern standard Arabic, even though that formal level is used in print, literature, scholarship, and, in modified form, on broadcast media.
One way to overcome the challenge is to give more classroom time to the dialects, which are notably easier to learn than modern standard Arabic, and to put them at the heart of college training. At colleges where Arabic is offered, dialects are generally taught as ancillary courses, or as incidental to the formal language. Few programs in the United States do more than offer an occasional class, or more than a single dialect (usually Egyptian, sometimes Levantine).
This should change, and there are signs that it can.
The flagship Arabic program at the University of Texas at Austin recently rethought the separation of formal and colloquial Arabic, and now mingles the two more integrally. And the new third edition of Georgetown University Press’s widely used textbook Al-Kitaab puts a variety of dialects at the center.
When American educators embrace fully the understanding that Arabic is a living language and one we need to learn to converse in, we may start to move beyond the American-century logic of one-way conversation embedded in Henry Luce’s essay. After the American century, the next generation of college students — and the citizens they will become — can help us listen to and engage a major portion of the world crucial to the future.
Brian T. Edwards is an associate professor of English and comparative literary studies and founding director of the program in Middle East and North African studies at Northwestern University. His newest book, After the American Century: The Ends of U.S. Culture in the Middle East, is forthcoming from Columbia University Press in the fall.