Personal tools
You are here: Home Publications GUIDE TO EVALUATION OF INSTRUCTION Literature Review

Literature Review

Table of Contents | Previous Section


There is a vast body of work regarding the evaluation of instruction.  The following items are references which have been found most useful, or which have been influential in shaping the discussion of evaluating teaching.

Definitions/Theories of teaching effectiveness
Suggested Readings:

Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

Cashin, W. E. 1989. "Defining and evaluating college teaching." (IDEA Paper No. 21). Manhattan: Kansas State University, Center for Faculty Evaluation and Development

Cohen, P.A. 1980. "Effectiveness of student rating feedback for improving college instruction: a meta-analysis of findings." Research in Higher Education, 13, 321-341.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Marsh, H. W. & Roche, L.A. 1997. "Making students’ evaluation of teaching effectiveness effective: the critical issues of validity, bias, and utility."  American Psychologist 52(11): 1887-1197.

McKeachie, W. 1997.  "Student ratings; the validity of use."  American Psychologist  v52, no.11, 1218-1225.

McKeachie, W. and Svinicki, M. 2006. McKeachie’s Teaching Tips. Boston, MA: Houghton Mifflin Company.

Ory, J. 2000. "Teaching evaluation: past, present, and future." New Directions for Teaching and Learning, no.83, Fall 2000

The criterion for teaching effectiveness varies depending on the goals of individual instructors and the mission of their institution.  In an effort to address the multi-dimensionality of teaching goals and responsibilities, teaching effectiveness is typically defined in terms of student learning (Hobson & Talbot, 2001).  However, focusing on student learning exacerbates the difficulty of defining teaching effectiveness, because learning assessment encompasses equally broad criterion that range from personal development, to increased knowledge, to self-discipline, to career development.  Marsh (1983) identified nine factors related to teaching effectiveness: learning/value, instructor enthusiasm, organization/clarity, group interaction, individual rapport, breadth of coverage, examinations /grading, assignments/readings, and workload/difficulty.

While research supports Marsh’s multi-dimensional factor analysis, a recent outcome assessment movement supports quantifying student learning in terms of job placement outcome (Ory, 2000).  This approach attempts to take the onus of evaluation-based decisions off student rating systems, which critics claim are too subjective.  Some of the subjectivity in the evaluation process results from students rating areas of teaching effectiveness that they are unable to access, which they identify as, 1) the goals, content, and organization of course design; 2) methods and materials used in delivery; and 3) evaluation of student work, including grading practices (Cohen, 1980; Cashin, 1989; McKeachie, 1997; 2006; Bain, 2004).


Student Perspective on Teaching Evaluations
Suggested Readings:

Chen, Y. & Hoshower, L.B. 2003. "Student evaluation of teaching effectiveness: an assessment of student perception and motivation." Assessment and Evaluation in Higher Education 28(1): 71-88.

Sojka, J., Gupta, A.K. & Deeter-Schmetz, D.R. 2002. "Student and faculty perception of student evaluations of teaching: a study of similarities and differences." College Teaching 50(2): 44-49.


Research shows that students tend to take teaching evaluations more seriously than faculty and institutional members commonly believe.  Students are more willing to participate and offer meaningful feedback when they believe and can see that their input is being considered and incorporated by their instructors and the institution.  In general, however, students do not perceive that their feedback is being used.  Some studies show that students place most value on evaluations for formative purposes, but research also indicates that students believe their input should be considered for summative purposes.  Students would like to see more specific items related to teaching effectiveness on student evaluation of teaching instruments.  (Sojka & Deeter-Schmetz, 2002; Chen & Hoshower, 2003)


Faculty Perspective on Teaching Evaluation
Suggested Reading:

Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

Gallagher, T.G. 2000. "Embracing student evaluations of teaching: a case study." Teaching Sociology 28, 140-147.

Hativa, N. 1995. "The department-wide approach to improving faculty instruction in higher education: a qualitative evaluation." Research in higher education 36(4): 377-413.

Sojka, J., Gupta, A.K. & Deeter-Schmetz, D.R. 2002. "Student and faculty perception of student evaluations of teaching: a study of similarities and differences." College Teaching 50(2): 44-49.


Traditionally, faculty have been very skeptical of teaching evaluations and have previously opposed such practices.  These negative feelings often spring from fear that student ratings will be used or misused for summative decision-making purposes.  Moreover, faculty often believe that students do not take evaluations seriously and that ratings encourage grade leniency.  Nonetheless, most faculty do pay attention to student feedback.  Further, when evaluations are used for formative purposes, instructors show a high degree of motivation to improve their teaching based on student input.  Studies have emerged showing how institutions and individual faculty members have begun using evaluations, consultations, and portfolios to qualitatively improve instruction.  When faculty are well informed about the purposes of evaluation, much of their anxiety dissipates and willingness to learn from student feedback increases.  (Sojka, Gupta, & Deeter-Schmetz, 2002; Hativa, 1995; Gallagher, 2000; Bain, 2004)


Summative Uses for Teaching Evaluations
Suggested Readings:


Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Hoyt, D.P. & Pallett, W. H.  1999. "Appraising teaching effectiveness: beyond student ratings."  IDEA Paper # 36

Kulik, J.A. 2001. "Student ratings: validity, utility, and controversy." New Directions for Institutional Research 109, 9-25.

McKeachie, W. 1997.  "Student ratings; the validity of use."  American Psychologist v52, no.11, 1218-1225.

McKeachie, W. and Svinicki, M. 2006. McKeachie’s Teaching Tips. Boston, MA: Houghton Mifflin Company

Theall, M. & Franklin, J. 2001. "Looking for bias in all the wrong places: a search of truth or a witch hunt in student ratings of instruction?" New Directions for Institutional Research 109: 45-56.


Teaching evaluations are commonly considered for summative purposes, including tenure, merit increase, retention for non-tenured faculty, promotion, and course assignment decisions.  While research generally agrees that teaching evaluations can be used in an effective and meaningful way to inform these decisions, often such data are misused, misinterpreted, or overused.  Some institutions use student ratings data as the sole criterion for evaluating teaching effectiveness, and moreover, these institutions often use only global items on student ratings forms to construct their evaluation.  Such misuse can breed distrust between faculty and administrators, resentment on the part of instructors for evaluations, and hinder other formative uses of these data.

Instead, researchers recommend that when such data are going to be used for summative purposes, various stakeholders, including administrators, faculty and students, should collaborate in determining a proper evaluation system.  The focus of evaluation should be on desired educational outcomes and whether these outcomes are being met.  If student ratings forms are to be used, the instruments must be subjected to rigorous validity tests and analysis.  Further, student rating data should be used in combination with other criteria in order to provide a better assessment of teaching, which is inherently a multidimensional construct.  Multiple sources of data collected over a span of time and covering a variety of courses taught would be most effective in informing summative decision-making. (Hobson & Talbot, 2001; Hoyt & Pallett, 1999; Theall & Franklin, 2001; Kulik, 2001; McKeachie, 1997; 2002; Bain, 2004)


Formative Uses of Teaching Evaluations
Suggested Readings:

Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

Gallagher, T.G. 2000. "Embracing student evaluations of teaching: a case study." Teaching Sociology 28, 140-147.

Hativa, N. 1995. "The department-wide approach to improving faculty instruction in higher education: a qualitative evaluation." Research in higher education 36(4): 377-413.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know. College Teaching 49(1): 26-31.

Hoyt, D.P. & Pallett, W. H.  1999. "Appraising teaching effectiveness: beyond student ratings."  IDEA Paper # 36

Johnson & Ryan 2000.  "A comprehensive approach to the evaluation of college teaching."  New Directions for Teaching and Learning, no.83. Fall 2000

Kulik, J.A. 2001. "Student ratings: validity, utility, and controversy." New Directions for Institutional Research 109, 9-25.

Theall, M. & Franklin, J. 2001. "Looking for bias in all the wrong places: a search of truth or a witch hunt in student ratings of instruction?" New Directions for Institutional Research 109: 45-56.


Using evaluations to inform instructors of their teaching effectiveness and to aid them in improving or enhancing their teaching constitute the formative purposes of teaching evaluations.  When used to inform teaching practices, specific dimensions of teaching must be identified and focused upon in order to bring about change.  Research indicates that evaluations are most effective in improving teaching when faculty members understand and value the importance of such processes, and an institutional and departmental culture that supports and respects teaching is evident. 

In particular, studies also indicate that mid-semester evaluations and feedback accompanied with consultation from a faculty developer or peer are more effective than traditional practices that leave the instructor to interpret end of semester findings by him/herself.  Evaluation systems for formative purposes often encompass more than just student ratings of teacher effectiveness.  Institutions have begun using portfolios, peer observation, self-review, and more qualitative approaches to improve teaching.  Similarly, recent establishment of faculty development centers on many campuses reveals a trend toward investing in the formative uses of evaluations.  (Hobson & Talbot, 2001; Hoyt & Pallett, 1999; Theall & Franklin, 2001; Kulik, 2001; Gallagher, 2000; Johnson & Ryan, 2000; Hativa, 1995; Bain, 2004)


On-Line Evaluations
Suggested Readings:

Johnson, T. 2002. "Online student ratings: will students respond?" AERA, New Orleans, LA, April 1-5, 2002.

Sorenson, D.L. & Johnson, T.D. 2003. "Online student ratings of instruction."   New Directions for Teaching and Learning, no. 96. 


Many institutions are moving to using on-line student ratings forms due to the ease of administration and affordability of this format of evaluation.  However, institutions are concerned about the lowered response rates these forms tend to generate.  Research shows that response rates can be bolstered through faculty encouragement, reward, and requirement of student completion of forms.  On-line forms tend to increase the likelihood that students will offer written comments to the open-ended questions.  (Johnson, 2002)


Organizational Structure for Evaluation Systems
Suggested Readings:

Arreola, Raoul. 2000. Developing a Comprehensive Faculty Evaluation System: A Handbook for College Faculty and Administrators on Designing and Operating a Comprehensive Faculty Evaluation System. Bolton, MA: Anker Publishing Co., Inc.

Hativa, N. 1995. "The department-wide approach to improving faculty instruction in higher education: a qualitative evaluation." Research in higher education 36(4): 377-413.

Seldin, P. et al. 1999. Changing Practices in Evaluation Teaching: a Practical Guide to Improved Faculty Performance and Promotion/Tenure Decisions. Bolton, MA: Anker Publishing Company, Inc. 


Institutions structure and locate their evaluation systems differently on their campuses.  In general, institutions either organize evaluations through a central office on campus or within individual departments.  With regard to the formative purposes of evaluation, the organizational location of evaluation systems can potentially influence the effectiveness of those systems.  In particular, certain researchers advocate a departmental approach to evaluation.  Advantages with this approach are: instruction and its improvement are often discipline specific and faculty most closely identify with their department and are more likely to seek guidance and information within their department. (Hativa, 1995) 


Consultation
Suggested Readings:

Cohen, P.A. 1980. "Effectiveness of student rating feedback for improving college instruction: a meta-analysis of findings." Research in Higher Education, 13, 321-341.

Marsh, H. W. & Roche, L. (1993). "The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness."  American Educational Research Journal, 30 (1), 217-251.

Murray, H. (1984). "The impact of formative and summative evaluation of teaching in North American universities." Assessment and Evaluation in Higher Education, 9 (2), 117-132.

Seldin, P. et al. (1999). Changing Practices in Evaluation Teaching: a Practical Guide to Improved Faculty Performance and Promotion/Tenure Decisions. Bolton, MA: Anker Publishing Company, Inc. 


The preponderance of evidence regarding instructional improvement based on the results of student evaluations of teaching shows that consultation plays a major role in implementing positive change.  While many institutions are incorporating professional teaching consultants into their administration, speaking with peers, teaching assistants, and students also benefit instruction (Cohen, 1980; Murray, 1984; Marsh & Roche, 1993; Seldin, 1999).


Other Forms of Evaluation
Suggested Readings:

Angelo, Thomas A., and K. Patricia Cross. 1993. Classroom Assessment Techniques: A Handbook for College Teachers, Second Edition. San Francisco, CA: Jossey-Bass Inc.

Davis, B.G. 1993. Tools for Teaching. San Francisco, CA: Jossey-Bass

Duncan, D. 2005. Clickers in the Classroom. Upper Saddle River, NJ: Pearson Education   

Hativa, N. 1995. "The department-wide approach to improving faculty instruction in higher education: a qualitative evaluation." Research in higher education 36(4): 377-413.

Hoyt, D.P. & Pallett, W. H.  1999. "Appraising teaching effectiveness: beyond student ratings."  IDEA Paper # 36

Johnson & Ryan 2000.  "A comprehensive approach to the evaluation of college teaching."  New Directions for Teaching and Learning, no.83. Fall 2000


In addition to student rating forms assessing teaching effectiveness, institutions also employ practices like portfolios, peer review, consultation, self-reports, alumni ratings, document review, student focus groups and interviews, observation, and department review to evaluate teaching.  Research advocates using different forms for evaluating different teaching methods and contexts for teaching.  Employing different types of evaluation have helped better inform formative and summative decisions.

Research on the validity and reliability of these different forms of evaluation is scarce and especially in comparison to the amount of research available on the usefulness of student rating forms.  However, studies do recognize that these other forms of evaluation can provide more qualitative and comprehensive information but may nonetheless be biased.  In addition, these approaches require significant amounts of time and commitment on the part of the institution and faculty members.  Institutions are continuing to incorporate more of these methods, and research has begun examining the effectiveness and utility of these approaches as well. (Johnson & Ryan, 2000; Hoyt & Pallett, 1999; Hativa, 1995)


Student Evaluations of Teaching (SET): Evaluation Forms
Suggested Reading:

Abrami, P.C. & d’Apollonia, P.A. 1990. "Validity of student ratings of instruction: what we know and what we do not." Journal of Educational Psychology 82(2): 219-31.

Centra, J. A.  1993. Reflective Faculty Evaluations. San Francisco: Josey-Bass.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Hoyt, D.P. & Cashin, W.E. 1977. "Development of the IDEA system" (IDEA Tech. Rep. No.1). Manhattan: Kansas State University, Center for Faculty Evaluation and Development.

Marsh, H. W. 1983. "Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/course/instructor characteristics." Journal of Educational Psychology, 75, 150-166.


According to Hobson & Talbot (2001), the three most widely used student evaluation forms are published instruments based on specific and global, self-reported student rating questionnaires.  Each system attempts to measure teaching effectiveness in terms of student learning and may be used for either summative or formative purposes. 

The Instructional Development and Effective Assessment (IDEA) form was developed by Hoyt & Cashin in 1977 at the Center of Faculty Evaluation and Development, Kansas State University.  The form was most recently revised in 1998, but adheres to its original premise that students rate instructors on a series of questions addressing their perceptions of the instructor ability to teach and design courses.  The form is paper and pencil based and has 46 self-reported questions that are both specific and global in nature.

Student Instructional Report was first developed by the Educational Testing Service in 1971.  It was most recently revised in 1989 and is made up of “thirty-nine questions, plus space for responses to ten additional questions that may be inserted locally” (Centra, 1993, p.188).  It asks both specific questions regarding the ability and behavior of the instructor, as well as global questions regarding the overall value of the course.
 
The Student’s Evaluation of Education Quality (SEEQ) was first developed in 1976 by Marsh, who at the time was living in Australia.  It was revised in 1991 to reflect American usage of the English language.  The form uses a multi-dimensional approach to evaluation based on a nine-factor analysis.  It is made up of thirty-five questions that pertain to nine areas of teaching: learning/value, instructor enthusiasm, organization/clarity, group interaction, individual rapport, breadth of coverage, examinations /grading, assignments/readings, and workload/difficulty.  The instrument can be used for both summative and formative purposes, but Marsh cautions strongly that administrators must have sufficient expertise in the evaluation process if the summative purposes lead to personnel decisions (Abrami and d’Apollonia, 1990).


Other Widely Used Forms
Suggested Readings:

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.


Instructor and Course Evaluation System, created by the University of Illinois at Urbana, uses a cafeteria based model that allows instructors to choose from over four hundred globally based questions.  The form is computer-based and only recommended for formative use.

Student Instructional Rating Systems, created by Michigan State University in 1982, focuses on specific questions regarding instructor effectiveness, but includes one global question regarding the “general enjoyment of the course” (Hobson & Talbot, p.29).

Instructional Assessment System, created by the Educational Assessment Center at the University of Washington, uses both specific and global questions on several forms that are designed specifically for different class sizes. 


Myths About Evaluation
Suggested Readings:

Aleamoni, L.M. 1999 "Student rating myths versus research facts from 1024 to 1998." Journal of Personnel Evaluation in Education 13(2): 153-166.

Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Kulik, J.A. 2001. "Student ratings: validity, utility, and controversy." New Directions for Institutional Research 109, 9-25.

McKeachie, W. and Svinicki, M. 2006. McKeachie’s Teaching Tips. Boston, MA: Houghton Mifflin Company.

Theall, M. & Franklin, J. 2001. "Looking for bias in all the wrong places: a search of truth or a witch hunt in student ratings of instruction?" New Directions for Institutional Research 109: 45-56.


Many myths exist about the usefulness of student ratings.  Some of these myths originate from faulty research studies, conflicting findings within the research literature, or reluctance on the part of administrators and faculty to evaluate and be evaluated, respectively.  Some common myths of student evaluations of teaching include: students are not able to make informed and consistent judgments about their instructors; student ratings are essentially a popularity contest; students cannot make accurate judgments unless they have been away from the course for a while; student ratings are negatively related to student learning; student ratings are based upon expected grade in course.

While the above myths have been adequately disproved by research, some criticisms of SET’s have been long-standing and not resolved.  These criticisms center on issues of validity and reliability, and factors that may bias teaching evaluations, including, student, course, and instructor characteristics. (Hobson & Talbot, 2001; Aleamoni, 1999; Theall & Franklin, 2001; Kulik, 2001; McKeachie, 2006; Bain, 2004)


Reliability of SETs
Suggested Readings:

Aleamoni, L.M. 1999. "Student rating myths versus research facts from 1024 to 1998." Journal of Personnel Evaluation in Education 13(2): 153-166.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Marsh, H. W. & Roche, L.A. 1997. "Making students’ evaluation of teaching effectiveness effective: the critical issues of validity, bias, and utility." American Psychologist 52(11): 1887-1197.


Reliability refers to the consistency of ratings among different raters and also the stability of such ratings over time.  Research has shown that student ratings show an acceptable level of consistency, or inter-rater reliability, given a class size of at least 15.  The level of consistency among raters increases as class size increases.  Longitudinal studies and studies of alumni ratings of an instructor/course have found that ratings show high levels of stability over time.  Further, cross-sectional studies show that student ratings reliably reflect instructor versus course effectiveness. (Hobson & Talbot, 2001; Aleamoni, 1999; Marsh & Roche, 1997)


Validity of SETs
Suggested Readings:

Aleamoni, L.M. (1999) "Student rating myths versus research facts from 1024 to 1998." Journal of Personnel Evaluation in Education 13(2): 153-166.

Hobson, S. M. & Talbot, D. M. 2001. "Understanding student evaluations: what all faculty should know." College Teaching 49(1): 26-31.

Greenwald, A. G. & Gillmore, G.M. 1997b. "Grading leniency is a removable contaminant of student ratings." American Psychologist 52(11): 1209-1217.

Greenwald, A.G. 1997. "Validity concerns and usefulness of student ratings of instruction." American Psychologist 52(11): 1182-1186

Kulik, J.A. 2001. "Student ratings: validity, utility, and controversy." New Directions for Institutional Research 109, 9-25.

McKeachie, W. 1997.  "Student ratings; the validity of use."  American Psychologist v52, no.11, 1218-1225


While researchers may argue about the degree of validity of SET’s, most find that SET’s are valid and can be used as a meaningful source of student input.  The validity of ratings is difficult to confirm, however, as there is no agreed upon definition of what constitutes effective teaching.  As a result, most researchers have adopted a construct validation or convergent validity approach, which involves examining the correlation between student ratings and other partial measures of teaching effectiveness, such as student learning, observer, peer, and alumni ratings.  Studies in this vein have reported generally positive and significant correlations between these measures and student ratings.  In addition, research on the validity of ratings has aimed to identify potential biasing factors not directly related to teaching effectiveness. (Hobson & Talbot, 2001; Aleamoni, 1999; Kulik, 2001; Greenwald, 1997; Greenwald & Gillmore, 1997b; McKeachie, 1997)


Potential Biasing Factors of SETs
Suggested Readings:

Aleamoni, L.M. 1999. "Student rating myths versus research facts from 1024 to 1998." Journal of Personnel Evaluation in Education 13(2): 153-166.

d’Apollonia & Abrami 1997.  "Navigating student ratings of instruction."  American Psychologist v52, n11, 1198-1208. 

Marsh, H. W. & Roche, L.A. 1997. "Making students’ evaluation of teaching effectiveness effective: the critical issues of validity, bias, and utility." American Psychologist 52(11): 1887-1197.

McKeachie, W. 1997.  "Student ratings; the validity of use."  American Psychologist v52, no.11, 1218-1225.

Theall, M. & Franklin, J. 2001. "Looking for bias in all the wrong places: a search of truth or a witch hunt in student ratings of instruction?" New Directions for Institutional Research 109: 45-56.


Researchers and critics of SET’s have suggested numerous factors which may bias student ratings of teacher effectiveness including: class size, grade leniency, instructor personality, gender, course workload, time that class meets, and type of class, including the academic discipline and required/elective status of class.  For each of these factors, research has been somewhat inconclusive, with some studies asserting a positive, negative, or null relationship between variables. Understanding the potential relationships, however, institutions and researchers have begun controlling for certain student and course characteristics before examining student ratings. (Aleamoni, 1999; Theall & Franklin, 2001; Marsh & Roche, 1997; d’Apollonia & Abrami, 1997; McKeachie, 1997)


Grading Leniency
Suggested Readings:

Chambers, B.A. & Schmitt, N. 2002. "Inequity in the performance evaluation process: how you rate me affects how I rate you." Journal of Personnel Evaluation in Education 16(2): 103-112.

Greenwald, A. G. & Gillmore, G.M. 1997a. "No pain, no gain? The importance of measuring course workload in student ratings of instruction." Journal of Educational Psychology 89(4): 743-751.

Greenwald, A. G. & Gillmore, G.M. 1997b. "Grading leniency is a removable contaminant of student ratings." American Psychologist 52(11): 1209-1217.

Marsh, H. W. & Roche, L.A. 1997. "Making students’ evaluation of teaching effectiveness effective: the critical issues of validity, bias, and utility." American Psychologist 52(11): 1887-1197.

McKeachie, W. 1997.  "Student ratings; the validity of use."  American Psychologist v52, no.11, 1218-1225. 


Many researchers have focused on the positive relationship between grades and student ratings as a potential biasing factor. The danger with this relationship is that it may encourage faculty to inflate their grades for the purpose of garnering higher ratings. Several theories exist that posit why a grades-rating relationship exists.  Prominent among them are: students infer course quality and own ability from received grades; students give high ratings in appreciation for lenient grades; and teaching effectiveness influences both grades and ratings.  The last of these, also termed the validity hypothesis, finds that effective teachers promote student learning (a measure of effective teaching) which results in higher grades.  Studies, however, examining differences in ratings and grades across instructor and course have been used to support each of the theories posited above. (Marsh & Roche, 1997; Greenwald & Gillmore, 1997a; McKeachie, 1997; Chambers & Schmitt, 2002; Greenwald & Gillmore, 1997b)


Global Versus Specific Items
Suggested Readings:

Bain, K. 2004. What the Best College Teachers Do. Cambridge, MA: Harvard Press.

d’Apollonia & Abrami. 1997.  "Navigating student ratings of instruction."  American Psychologist v52, n11, 1198-1208. 

Gallagher, T.G. 2000. "Embracing student evaluations of teaching: a case study." Teaching Sociology 28, 140-147.

Young, S. Y. & Shaw, D. G. 1999. "Profiles of effective college and university teachers." The Journal of Higher Education 70(6): 670-686.


Student rating forms generally contain both global (or overall rating) items and specific items, which assess specific aspects of the instructor and course.  Research is split on the value of both of these types of items.  Some argue that teaching is multi-dimensional and therefore requires specific items to accurately assess different facets of teaching.  Others show that when specific items are factor analyzed, they essentially reduce down to one or two items that are global in nature.  Studies also reveal that responses on specific and global items are highly correlated. 

With regard to the uses of these types of items, researchers warn against making summative decisions based solely on ratings on global items.  In addition, formative purposes seem better informed by having data on specific areas that faculty can target in order to improve their teaching. (Gallager, 2000; d’Apollonia & Abrami, 1997; Young & Shaw, 1999; Bain, 2004)

top

Table of Contents | Previous Section

Document Actions
  • Print this
  • ShareThis