FAB4

Elyssa, Claire, Beth, Kaitlin

Teacher Evaluation Proposal
A. Instruction Guide for Teaching Evaluations
Overview
This appraisal process will utilize a job analysis for a professor at the postsecondary level
to determine relevant dimensions to consider when measuring job performance. It will then use
several methods for assessing these dimensions, including student and peer evaluations. The
appraisal will not require evaluators to formally collect performance information; however, at the
beginning of the semester, professors will be made aware of the dimensions and of related
behaviors so that they may take note of applicable peer behaviors, if any, throughout the
semester. This process is committed to following legal and otherwise highly-recommended
guidelines to formulate objective, fair evaluations for those being assessed. It will also identify
possible errors pertaining to these types of evaluations, which will then prompt implementation
of probable solutions for these errors.
Evaluation Process and Determinants of Job Success
Dispensation of evaluations will occur at the end of each semester, whereby student
evaluators will submit one appraisal for each of his/her professors. Therefore, each professor will
receive appraisals by all of his/her students in every one of his/her classes once per semester
towards the end of the semester. Instructions on this sheet will inform students about the
importance of professor evaluations, urging them to make their appraisals as accurate as possible
and using wording that encourages objective evaluations of specific behaviors rather than
personality traits. More specifically, students will fill out a questionnaire harnessing a graphic
rating scale with behavioral definitions. This method was chosen because the graphic rating
scale, which is the most common method for evaluation, is among the easiest to administer to a
large number of respondents (Muchinsky, 2012).
This particular scale will range from 1 (strongly disagree) to 6 (strongly agree) based on
how much the student believes the professor to have exhibited the behavior. The evaluation will
consist of eight unmarked sectionsone for each dimension, as identified by a job analysis of
important knowledge, skills, attitudes, and other competencies for this position. For more
information on selected dimensions, see Criteria within the section Teacher Evaluation. Each
section will consist of two or three behaviors that compose the specified dimension, and
operational definitions will be given for each behavior to maximize objectivity (see Appendix A).
The use of several behaviors to measure each dimension will allow for multiple aspects of one
dimension to be assessed. Professors should be rated at the level 5 (agree) or 6 (strongly agree)
for at least 75% of statements to be considered acceptable in regards to job performance.
This appraisal process will also use peer ratings, in which group members rate other
members performances on selected dimensions (Muchinsky, 2012). This particular usage of peer
ratings is characterized by divisions by level, in which only the same type of faculty will
evaluate each other. For example, tenured professors will evaluate other tenured professors,
while associate professors will evaluate other associate professors. This will decrease the
potential for confounds resulting from task differences that may exist with staff members at the
same organizational level (Muchinsky). Additionally, to protect against potential bias, two staff
members will rate one peer so that interrater reliability can be determined.

FAB4
Elyssa, Claire, Beth, Kaitlin
A peer rating system was selected due to its high predictive validity (.4 to .5) and
reliability (.8 to .9), as noted by Muchinsky. The specific rating system being utilized for this
process will be a Behavioral Observation Scale (BOS), which is characterized by a graphic rating
scale to determine the frequency of critical incidents. A critical incident is a behavior serving as a
crucial determinant in whether or not a performance is good or bad (Muchinsky, 2012).
Similar to the student evaluation, there will be eight unmarked sections (one for each dimension)
and each will list two or three behaviors that have been identified as critical incident behaviors
for good performance. Collectively, these incidents must correspond to the same or very
similar aspects measured by the behaviors in the student evaluations, which will allow for later
comparisons for validity purposes. However, the scale will range from 1 (never observed) to 6
(very frequently observed). As before, professors must exhibit 75% of the listed behaviors at the
level 5 (often observed) or 6 (very frequently observed) to be considered acceptable in regards
to job performance.
Further Process-related Recommendations and Practices
For legal defense purposes, Muchinsky (2012) argues that the appraisal process should be
uniform and standardized, the criteria should be formally communicated during training, and
employers should provide notice of deficiencies (as well as opportunities to correct them).
Finally, employees should have access to the results, there should be multiple, diverse raters, and
written instruction should be clearly provided during training and on the evaluation forms.
Together, these components will serve to defend the validity and objectivity of evaluations, if
legally challenged.
This appraisal has taken several steps to comply with these recommendations. Firstly, the
overall process and evaluations remain the same from professor to professor, thus making the
assessment standardized and uniform. The instructions and overall process are formally
communicated through training, as well as at the beginning of the year when teachers are
informed of the criteria for which they will be evaluated. Through the utilization of 360-degree
feedback (see Identifying and Resolving Evaluation Errors), this process has also accounted for
the recommendation to give notice of deficiencies and ample opportunity for correction.
Additionally, the results will be made available to employees in a timely manner once
assessments have been completed, collected, and reviewed, which also serves as a way of
notifying faculty of deficiencies highlighted in the evaluations.
While it may prove difficult to find diverse raters, the student evaluators will exhibit
some diversity, and through the collection of both student and peer evaluations there will be
multiple raters, which will provide more diversity than other rating systems. Additionally,
training and the use of raters from multiple positions, i.e., subordinates (students) and peers
(fellow professors), helps to reduce certain types of bias. More specifically, it should protect
against the threat of bias from students that dislike the professor and, according
to Muchinsky (2012), from peers that consider them friends. Lastly, this process has accounted
for the necessity of written instructions for training raters through providing clear instructions
both on the evaluations and during training.
It should also be noted that this process protects against other types of bias that may threaten
its validity and objectivity as well. For instance, societal expectations for women to display
altruistic (e.g., helpful) behavior in the workplace suggests that women will suffer greater
negative consequences than men when they fail to do so (Heilman & Chen, 2005), and
evaluations may be moderated by race of the rater and/or ratee (Stauffer & Buckley, 2005). To

FAB4
Elyssa, Claire, Beth, Kaitlin
counter these biases and others, this appraisal uses multiple methods to identify potential biases
and concrete behavioral observations to restrict subjectivity as much as possible.
Identifying and Resolving Evaluation Errors
According to Muchinsky (2012), performance evaluations may be affected by any and all
of three possible rating errors: halo, leniency, and central tendency errors. A halo error refers to
the evaluations being influenced by the raters own biases or opinions about the employee. As
Muchinsky notes, this is considered the most serious and pervasive rating error. More
specifically, evaluators should be most concerned with possibility of an invalid halo error,
characterized by the raters apparent failure to differentiate an employees performance across
dimensions (Muchinsky, pp.213 214). In contrast, a leniency error is one in which the rater is
consistently easy (positive leniency) or harsh (severity/negative leniency) in their ratings
(Muchinsky). The concern is that evaluations are either positively or negatively skewed from
their true level of ability, and therefore they are inaccurate measures of a ratees actual
capabilities. Central tendency errors occur when the rater is unwilling to assign high or low
ratings, resulting in evaluations that typically fall along the middle of a scale (Muchinsky).
Unfortunately, this also fails to differentiate employees performance.
In order to reduce rating errors, training processes can be implemented. Namely, all the
evaluators should be educated on the purpose of performance appraisals and on the different
types of rating errors and rater biases that could occur. At the beginning of each year, teacher
evaluators should also be specifically trained and educated to the scale that they will use. By
increasing awareness about issues and biases pertaining to evaluations, training should reduce
reporting errors and evaluator bias. Similarly, utilizing frame-of-reference training can help
evaluators calibrate their appraisals. This type of training provides examples of good, bad, and
average performances, in addition to feedback on the accuracy of evaluations (Muchinsky,
2012). Consequently, these experiences provide references that encourage standardization of
evaluations across all raters, which is typically most impactful for positive leniency errors
(Muchinsky). Overall, for training to be most effective, it should highlight how it is relevant to
and beneficial for successful (e.g., accurate) appraisals, and trainees should receive experiences
that are realistic previews for situations that they may encounter (Salas et al., 2012).
Additionally, to reduce central tendency errors, evaluations will omit responses allowing for
neutral opinions/observations (e.g., no opinion).
Finally, to further eliminate central-tendency errors and leniency errors, this appraisal
will implement an employee-comparison method, which allows for job performance to be
compared to that of other employees instead of a defined standard (Muchinsky, 2012). One such
method involves the utilization forced distribution. As Muchinsky explains, this technique
assumes performances to be normally distributed and therefore divides a normal distribution
curve into five to seven categories for each specified dimension. Raters evaluate employees by
placing them into the categories based on evaluations, and each category has a predetermined
percentage of people to be placed there (e.g., 40% of ratees may be placed in the category
directly under the distribution peak), which forces a normal distribution among the ratees. A
variant of top-grading should also be implemented in that the bottom 10% of ratees will receive
an opportunity for 360-degree feedback. That is, rather than the typical rank and yank

FAB4
Elyssa, Claire, Beth, Kaitlin
technique, which fires the bottom 10% of a forced distribution, this proportion of professors will
evaluate their own performance, in addition to receiving performance evaluations from
subordinates, supervisors, and peers (Muchinsky). It is the hope that this feedback will facilitate
self-development by raising awareness about any discrepancies in their performance.
Determining Validity
This process utilizes multiple appraisal methods and raters to determine the validity of
the ratings provided. By correlating judgment data from performance appraisals with those
obtained from additional methods (e.g.., from peer ratings), it is possible to assess the
accuracy of these evaluations (Muchinsky, 2012). Therefore, this process involves correlating the
appraisal outcomes between student and peer evaluations in terms of whether or not job
performance has been deemed acceptable. The higher the resulting correlation coefficient, the
more validity this process holds (Muchinsky).
Additionally, validity will be achieved through the control for biases, such as those based
on friendship (from peer raters) and/or from disdain towards professors (from students). As
aforementioned, students will utilize a graphic rating scale in terms of agreement with behavioral
statements/definitions, while peers will rate the incidence of crucial behaviors serving as
determinants of good performance. It is therefore likely that evaluations will contain slightly
different content; however, both evaluations measure the same constructs, and so outcomes of
both of these surveys should then be correlated. In this way, we hope to achieve validity.

B. Teaching Evaluations
Criteria
Both student and teacher evaluations will be measuring the following dimensions, each with
specific behavioral criteria:
Content Knowledge: knowledge of the field of practice, including knowledge of
current research and any specialty content related to area of expertise.
Quality Instruction: ability to identify the educational needsof others and to develop
appropriate course materials and methods to instruct students in effective ways.
CommunicationSkills: use of effective methods for the sharing knowledge and opinions
through written work and speech to a variety of populations.
Time Management: development of specific goals and plans to successfully prioritize,
organize, and accomplish specifictasks by designated deadlines.

FAB4
Elyssa, Claire, Beth, Kaitlin
Research: ability to identify underlying principles or relationships, in addition to
systematicallybreakingdown data from studies into separate components and then
conducting careful experimentation and analysis of each component.
Learning Strategies: knowledge of different methods of teaching and learning, which
includes the utilization of various activities and projects to promote critical thinking,
creativity, and active student engagement with/learning of the course material through all
major learning styles.
Education and Training: adequate experience with curriculum and course design, in
addition to knowledge of appropriate methods for instructing and evaluating groups and
individuals.This also includes the ability to provide timely feedback to the students and
opportunities for remediation in order to foster improvement and higher-level learning.
Student-Orientation: the inclination to engage with students frequently in a positive
manner and to create an environment that will best accommodate all students and their
needs.
Addressing Content-Related Recommendations
Muchinsky (2012) makes several recommendations on how to address the content of
performance appraisals, such as being objective, making sure that all criteria upon which the
subject is being rated are derived from a job analysis, ensuring that criteria are based on
behaviors and not on personality traits, ensuring that all criteria are within the locus of control of
the ratee, and relating criteria to the specific functions of their job rather than global assessments.
This appraisal uses several methods to comply with these suggestions. Objectivity is
achieved through the analysis of concrete behaviors because it does not provide room for
subjective opinions to be included. Additionally, incorporating comparison with peer evaluations
will reduce and identify the impact of subjective ratings given by students that dislike the
professor and/or peers that are also friends. All listed behaviors pertain to dimensions that were
chosen by a job analysis, and therefore they have been identified as directly pertinent to
successful job performance in this position. All items that appear on the survey are assessing
observable behaviors (e.g., actions) of the professor, rather than traits. The behaviors that are
being assessed are also directly controlled by the ratee, and this analysis does not focus on
behaviors that may be significantly impacted by environmental factors. Finally, all behaviors
measure specific aspects for job functioning rather than overall job components.

C. Explanation of Performance Appraisal

This appraisal uses a combination of a graphic rating scale with behavioral definitions and a
Behavioral Observation Scale as the two forms of evaluation. It administers the graphic rating
scale to the students because it is the easiest to administer to larger groups of people (Muchinsky,

FAB4
Elyssa, Claire, Beth, Kaitlin
2012). This appraisal also includes peer evaluations as a comparison with the student evaluations
because peer-rating systems have high predictive validity as well as high reliability (Muchinsky,
2012). Most importantly, however, this separate evaluation technique provides a measure to
correlate with student evaluation outcomes in order to determine validity.
For both forms, only specific behavioral components are included, which discourages
subjective evaluations. This use of concrete behavioral measures and comparisons also allows for
protection against biases that may skew evaluations. The objective criteria were chosen because
they are representative of good behavior, which measures the presence (or absence) of desirable
behaviors in the workplace and can therefore assess job performance most accurately.
Additionally, by including multiple behaviors/critical incidents for each dimension, this appraisal
is better-able to address a range of factors composing each dimension, especially those identified
by a job analysis as necessary for successful job functioning.

References
Heilman, M. E., & Chen, J. J. (2005). Same Behavior, Different Consequences: Reactions to
Men's and Women's Altruistic Citizenship Behavior. Journal of Applied Psychology,
90(3), 431-441. doi:10.1037/0021-9010.90.3.431
Muchinsky, P. M. (2012). Psychology Applied to Work (10th ed.). Summerfield, NC:
Hypergraphic Press.
Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The Science of
Training and Development in Organizations: What Matters in Practice. Psychological
Science in the Public Interest, 13(2), 74-101. doi:10.1177/1529100612436661
Stauffer, J. M., & Buckley, M. R. (2005). The Existence and Nature of Racial Bias in
Supervisory Ratings. Journal of Applied Psychology, 90(3), 586-591. doi:10.1037/00219010.90.3.586

FAB4
Elyssa, Claire, Beth, Kaitlin

Appendix A
Teacher Evaluation (Student Copy)
Reflecting on your professors behavior over the course of this class, please rate your
agreement with the following statements using this scale. These evaluations serve as crucial
feedback so that this university can improve everyones learning experience!
PLEASE MAKE YOUR REFLECTIONS AS ACCURATE AS POSSIBLE.

1
Strongly
disagree

2
Disagree

3
Somewhat
disagree

4
Somewhat
agree

5
Agree

6
Strongly
agree

Reflecting on my professors behavior while teaching this class, s/he:

_____ Is knowledgeable (explanations show clear and thorough understanding of the
information).
_____ Adequately answers questions (has sufficient knowledge to answer questions related to
his/her area of expertise, or to accurately surmise answers to extraneous questions to the
best of his/her ability).

FAB4
Elyssa, Claire, Beth, Kaitlin
_____ Presents appropriate course materials (provides course materials that adequately relate to
and teach course content).
_____ Gives articulate explanations (provides explanations that are well-thought-out and
successfully convey information in ways that resolve ambiguities in understanding).
_____ Clearly presents the information (effectively presents course information in ways that are
easy to follow and to comprehend, especially in regards to complex information).
_____ Uses concise grammar and written/oral instruction (instructions are easy to comprehend,
especially with regard to sentence structuring to facilitate comprehension of the
information).
_____ Clearly communicates course requirements (well-defined class expectations and layout, or
else timely warning of changes in course requirements/expectations).
_____ Clearly communicates homework/project requirements (well-defined homework/project
guidelines and expectations, or else timely warning of changes in
guidelines/expectations).
_____ Holds students to obtainable expectations and requirements (maintains expectations that
students are capable of meeting, including adequate deadlines for task completion and
spacing between projects, etc.)
_____ Is available (responds to inquiries within a reasonable time frame, holds consistent office
hours and/or is able to meet for extra help or for miscellaneous matters, etc.).
_____ Provides good feedback (provides timely and useful feedback on assignments, either
through grades or comments/critiques).
_____ Is well-informed (accounts for current research and updates in relation to the specific
content area).
_____ Uses accurate reporting (displays and facilitates sufficient critical-thinking when reporting
on research and potential limitations).
_____ Utilizes diverse instructional techniques (uses multiple instructional techniques
throughout the semester, such as lectures, videos, interactive media, group discussions,
interteaching, etc.).
_____ Accommodates different learning strategies (instructional techniques and explanations

FAB4
Elyssa, Claire, Beth, Kaitlin
reflect a multitude of teaching strategies to account for multiple learning types in
students).
_____ Provides adequate class structure/design (implements a course curriculum and design with
appropriate pacing and structure to best facilitate learning).
_____ Uses appropriate evaluation methods (uses evaluations that accurately and objectively
reflect student progress and effort in the course).
_____ Has well-organized instruction (presents information in a structured, logical manner that
facilitates deeper understanding and learning of the information).
_____ Exhibits student engagement (often converses with and engages individual and/or groups
of students through discussions, before/during/after classes, etc.).
_____ Is flexible (takes into account student feedback on classes/assignments, adjusts for
unforeseen personal situations with students, accommodates disabilities to the best of
his/her ability, etc.).
_____ Exhibits appropriate interactions (treats students with respect and dignity, is considerate
and respectful of student beliefs and values, etc.).