Online Course Evaluation

Literature Review and Findings

Prepared by:
Jessica Wode
Jonathan Keiser

Academic Affairs
Columbia College Chicago
Spring 2011

T ABLE

OF

C ONTENTS

Summary of Scholarly Research on Student Course Evaluations...............................................2

Recommendations for Improving Response Rates......................................................................6
Additional Reading on Online Evaluations: Annotated Bibliography...........................................7

S UMMARY

OF

S CHOLARLY R ESEARCH

ON

S TUDENT C OURSE E VALUATIONS

The validity and reliability of course evaluations

Researchers generally consider student evaluations of instructors to be highly
reliable and at least moderately valid.1,2,3
Other methods of evaluation (such as evaluations by colleagues or trained observers)
have not been found to be reliable and therefore not valid.1
Student ratings of instructors have been found to be related to ratings of instructors
skills in course organization, rapport with students, and fair grading; variance in
organizational skill (having an organized course plan and clearly identifying what
students need to do) explained most variance in student evaluations.4
Alumni rarely change their opinions of former teachers. 1,3
When instructors collect mid-term feedback from students and have an honest
discussion about it with someone, it leads to higher evaluations at the end of the
semester as well as higher final exam scores, providing evidence that good
evaluations can lead to better teaching.5
Although grades do have some effect on how students rate instructors,6 its effect is
fairly low7 and can be statistically adjusted for.8 Grades do not have as large of an
effect as do how much students feel theyve learned,9 how much they felt stimulated
by the class,10 and whether the class was appropriately difficult (courses are rated
lower for being too easy or too difficult).11
Contrary to the retaliation theory, students who do poorly in a class are equally or
less likely than those who do well to complete course evaluations.12

1 Centra, John. 1993. Reflective faculty evaluation: enhancing teaching and determining faculty effectiveness
(Jossey Bass higher and adult education series). San Francisco: Jossey-Bass Inc Pub.
2 Hobson, Suzanne M., and Donna M. Talbot. "Understanding Student Evaluations: What All Faculty Should
Know." College Teaching 49, no. 1 (2001): 26-31.
3 Aleamoni, Lawrence M. "Student Rating Myths Versus Research Facts from 1924 to 1998." Journal of
Personnel Evaluation in Education 13, no. 2 (1999): 153-166.
4 Jirovec, Ronald L., Chathapuram S. Ramanathan, and Ann Rosegrant-Alvarez. "Course Evaluations: What are
Social Work Students Telling Us About Teaching Effectiveness?" Journal of Social Work Education 34, no. 2
(1998): 229-236.
5 Overall, J.U., and Herbert W. Marsh. "Midterm Feedback from Students: Its Relationship to Instructional
Improvement and Students' Cognitive and Affective Outcomes." Journal of Educational Psychology 71, no. 6
(1979): 856-865.
6 Johnson, Valen E. "Teacher Course Evaluations and Student Grades: An Academic Tango." Chance 15, no. 3
(2002): 9-16.
7 Gigliotti, Richard J., and Foster S. Buchtel. "Attributional Bias and Course Evaluations." Journal of Educational
Psychology 82, no. 2 (1990): 341-351.
8 Greenwald, Anthony G., and Gerald M. Gillmore. "Grading leniency is a removable contaminant of student
ratings." American Psychologist 52, no. 11 (1997): 1209-1217.
9 Bard, John S. "Perceived learning in relation to student evaluation of university instruction." Journal of
Educational Psychology 79, no. 1 (1987): 90-91.
10 Remedios, Richard, and David A. Lieberman. "I liked your course because you taught me well: the influence
of grades, workload, expectations and goals on students' evaluations of teaching." British Educational
Research Journal 34, no. 1 (2008): 91-115.
11 Centra, John A. "Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course
Work? " Research in Higher Education 44, no. 5 (2003): 495-518.
12 Liegle, J O and D S McDonald. Lessons Learned From Online vs. Paper-based Computer Information
Students' Evaluation System. In The Proceedings of the Information Systems Education Conference 2004, v 21
(Newport): 2214. ISSN: 1542-7382.

Online vs. paper course evaluations

The one consistent disadvantage to online course evaluations is their low response
rate13,14; using reminder e-mails from instructors and messages posted on online
class discussions can significantly increase response rates.15
Evaluation scores do not change when evaluations are completed online rather than
on paper.14,16
Students leave more (and often more useful) comments on online evaluations
compared to paper evaluations.13,16,17
Students, faculty, and staff generally view online evaluations more positively than
paper evaluations.13,16

Anderson, Heidi M., Jeff Cain, and Eleanora Bird. "Online Student Course Evaluations: Review of Literature
and a Pilot Study." American Journal of Pharmaceutical Education 69, no. 1, article 5 (2005).
14 Avery, Rosemary J., W. Keith Bryant, Alan Mathios, Hyojin Kang, and Duncan Bell. "Electronic Course
Evaluations: Does an Online Delivery System Influence Student Evaluations?." Journal of Economic Education
37, no. 1 (2006): 21-37.
15 Norris, John, and Cynthia Conn. "Investigating Strategies for Increasing Student Response Rates to OnlineDelivered Course Evaluations." Quarterly Review of Distance Education 6, no. 1 (2005): 13-29.
16 Donovan, Judy, Cynthia E. Mader, and John Shinsky. "Constructive Student Feedback: Online vs. Traditional
Course Evaluations." Journal of Interactive Online Learning 5, no. 3 (2006): 283-296.
17 Kasiar, Jennifer B., Sara L. Schroeder, and Sheldon G. Holstad. "Comparison of Traditional and Web-Based
Course Evaluation Processes in a Required, Team-Taught Pharmacotherapy Course." American Journal of
Pharmaceutical Education 66 (2002): 268-270.
13

Student perceptions of course evaluations

Students tend to feel that evaluations have no effect on teacher performance, and
they dont seem to know if anyone other than the instructor sees the evaluations.18
Surveys of students typically indicate that students believe faculty and administrators
dont take their evaluations seriously.19 This may be justified, as some studies have
found that instructors do not view student evaluations as valuable for improving
instruction20 and very few report making changes to their courses as a result of
course evaluations.21
Students are more likely to complete course evaluations if they see value in them
(e.g., understand how they are being used, believe that their opinions have an
effect).22

Marlin, James W., Jr. "Student Perceptions of End-of-Course Evaluations." The Journal of Higher Education
58, no. 6 (1987): 704-716.
19 Spencer, Karin J., and Liora Pedhazur Schmelkin. "Student Perspectives on Teaching and its Evaluation."
Assessment & Evaluation in Higher Education 27, no. 5 (2002): 397-409.
20 Nasser, Fadia, and Barbara Fresko. "Faculty Views of Student Evaluation of College Teaching." Assessment &
Evaluation in Higher Education 27.2 (2002): 187-198.
21 Beran, Tanya N., and Jennifer L. Rokosh. "Instructors' perspectives on the utility of student ratings of
instruction." Instructional Science 37.2 (2009): 171-184.
22 Gaillard, Franklin D., Sonja P. Mitchell, and Vahwere Kavota. "Students, Faculty, And Administrators
Perception Of Students Evaluations Of Faculty In Higher Education Business Schools." Journal of College
Teaching & Learning 3, no. 8 (2006): 77-90.
18

Effects of allowing students access to course evaluation data

Students who do not have access to course evaluating ratings rate course
evaluations as more important to making a course selection than those who do have
access.23 This may indicate that students think course evaluation data will be more
helpful than it actually is.
If all else is equal, a student is twice as likely to choose an instructor with excellent
ratings over an instructor with good ratings; however, students are willing to select
a poor instructor if they believe they will learn a lot from the class.24
Students will choose a highly rated course over less highly rated courses even if the
workload is greater for that course than the others.25
Results are mixed on whether receiving evaluation information influences how
students consequently rate the instructor.24 Some studies have indicated that
students who receive information that an instructor was rated highly will rate that
instructor highly, and vice versa.26,27
Rulings involving the University of Wisconsin and University of Idaho found that
students had a right to view the results of student evaluations of faculty.28

23 Wilhelm, Wendy Bryce, and Charles Comegys. "Course Selection Decisions by Students on Campuses With
and Without Published Teaching Evaluations." Practical assessment, research & evaluation 9, no. 16 (2004).
http://PAREonline.net/getvn.asp?v=9&n=16 (accessed February 15, 2010).
24 Wilhelm, Wendy Bryce. "The Relative Influence of Published Teaching Evaluations and Other Instructor
Attributes on Course Choice." Journal of Marketing Education 26, no. 1 (2004): 17-30.
25 Coleman, Jeffrey, and W.J. McKeachie. "Effects of Instructor/Course Evaluations on Student Course
Selection." Journal of Educational Psychology 73, no. 2 (1981): 224-226.
26 Perry, Raymond P., R. Ronald Niemi, and Keith Jones. "Effect of prior teaching evaluations and lecture
presentation on ratings of teaching performance." Journal of Educational Psychology 66, no. 6 (1974): 851856.
27 Griffin, B.W. "Instructor Reputation and Student Ratings of Instruction." Contemporary Educational
Psychology 26.4 (2001): 534-552.
28 Haskell, R.E.. "Administrative Use of Student Evaluation of Faculty." Education Policy Analysis Archives 5, no.
6 (1997). http://epaa.asu.edu/ojs/article/view/622/744 (accessed February 22, 2010).

R ECOMMENDATIONS FOR I MPROVING R ESPONSE R ATES

The literature suggest that there are three primary methods to improve response rates on
end-of-course evaluations:
1) Make evaluation a part of the course (most effective)
2) Send reminder notices
2) Offer a small incentive
1. Make Evaluation Part of the Course
The most effective method to maintain high-quality response rates is to make evaluation
part of your course. Simply administering a mid-semester course evaluation and providing
the results and your plan of action based on their feedback to the class will dramatically
improve response rates at the end of the year.
This is because it addresses students' primary complaint about course evaluation: No one
looks or even cares about what I have to say about the course. Surveys and information
suggest that students have little confidence that faculty or administrators pay attention to
the results. If you show them that their feedback is important, studies show
that they will provide that feedback to you.
2. Send Reminder Notices
At Columbia, as part of the centrally administered option, three email reminders are sent to
the students through their university email accounts each week the evaluation is open.
There is also a pop-up reminder each time a student logs into OASIS.
Instructors are encouraged to remind their own students of the importance of the
evaluations and encourage their participation through whatever communication channel you
have established for your course.
3. Offer a Small Incentive
The literature stated that small incentives will boost the response rates from students.
Examples that were provided were one-half of one percent grade enhancement or contests
for prizes like an iPod.

A DDITIONAL R EADING

ON O NLINE E VALUATIONS : A NNOTATED B IBLIOGRAPHY

Note:ThisliteraturereviewisbasedonthebibliographyofCollings,D.&Ballantyne,C.S.(2004,November2425).
Onlinestudentsurveycomments:Aqualitativeimprovement?

Anderson,H.,Cain,J.,Bird,E.(2005)OnlineStudentCourseEvaluations:ReviewofLiterature
andaPilotStudy.AmericanJournalofPharmaceuticalEducation2005;69(1)Article5.

Theliteraturereviewrevealedseveralstudiesthatfoundnostatisticallysignificantdifferencesbetween
deliverymodes.Twoalsonotedthatstudentsprovidedmorecommentsintheonlineforms.Responserates
variedwidely.TheUniversityofKentuckyCollegeofPharmacy,drivenbythefacultysdesireformoretimely
returnofresults(34monthstypically),launchedapilotstudyofonlineevaluationsin3courses.Theresponse
ratesforthe3courseswere85%,89%,and75%.The9coursesusingthepaperformsaveragedan80%response
rate(consistentwiththe2previousyearsalsoabout80%).Thecommentsontheonlineformsweremore
frequentandlongerthanthepaperforms.Studentslikedtheonlineformbetterthanthepaperformand
thoughttheycouldprovidemoreeffectiveandconstructivefeedbackonline.

Anderson,J.,G.Brown,andS.Spaeth.(2006)OnlineStudentEvaluationsandResponseRates
Reconsidered.Innovate2(6).http://www.innovateonline.info/index.php?view=article&id=301

SynopsisfromInnovate:Manyadministratorsaremovingtowardusingonlinestudentevaluationstoassess
coursesandinstructors,butcriticsofthepracticefearthattheonlineformatwillonlyresultinlowerlevelsof
studentparticipation.JoanAnderson,GaryBrown,andStephenSpaethclaimthatsuchaconcernoftenfailsto
acknowledgehowtheevaluationprocessalreadysuffersfromsubstantiallackofengagementonthepartof
studentsaswellasinstructors;theonlineformat,theyassert,merelyinheritsthefundamentalproblemof
perceivedirrelevanceintheprocessitself.Afteraddressingthereasonsbehindthisproblemanddiscussing
howwelldesignedonlineevaluationscanstillmakeapositivedifference,theauthorsdescribethedevelopment
andimplementationofacomprehensive,collegewideonlineevaluationsurveyatWashingtonStateUniversity's
CollegeofAgricultural,Human,andNaturalResources.Inreviewingthesurveyresults,theyfoundthatclasssize,
academicdiscipline,anddistributionmethodplayedanegligibleroleinstudentresponserates.However,they
foundthatvariancesinresponserateweresignificantlyinfluencedbytherelativelevelofparticipationamong
facultymembersanddepartmentheadsintheoriginaldevelopmentofthesurvey.Theauthorsmaintainthat
onlinesurveyscanmaketheprocessmorerelevantandmeaningfultostudents,buttheyconcludethateliciting
greaterresponserateswillstillrequiresustainedsupport,involvement,andadvocacybyfacultymembersand
administrators.

Ardalan,A.,Ardalan,R.,Coppage,S.,andCrouch,W.(2007)Acomparisonofstudentfeedback
obtainedthroughpaperbasedandwebbasedsurveysoffacultyteaching.BritishJournalof
EducationalTechnology.Volume38Number62007.

Thispaperprovidesasummaryofthecurrentresearchinonlinevs.paperevaluationsaswellasresultsfroma
studenttocomparethefeedbackresults.Thesameformwasgivento46sectionpairingsonepaperandone
online.Theonlineresponseratewas31%(392outof1276possibleresponses)andthepaperwas69%(972
outof1415).Nosignificantdifferencewasfoundinthequantitativeratingsbetweenthetwomethods.They
examinedthedifferencesonanoveralleffectivenessquestioninratingforfacultywhowereabovethecollege
averageandthenforfacultywhowerebelowthecollegeaverage.Facultywhowereabovetheaveragewere
scoredslightlyloweronlineandthefacultywhowerebelowthecollegeaveragewerescoredhigheronline.
Therewasnosignificantdifferenceinthenumberofstudentsgivingopenendedfeedbackonline;however,
therewasasignificantincreaseinthelengthofopenendedfeedbackonline.

Avery,RosemaryJ.,BryantW.K.,Mathios,A.,Kang,H.,andBell,D.(2006).ElectronicCourse
Evaluations:DoesanOnlineDeliverySystemInfluenceStudentEvaluations?Journalof
EconomicEducation.Washington:Winter2006.Vol.37,Iss.1p2138(ProQuestdocumentID
973267691).

TheDepartmentofPolicyAnalysisandManagementaCornellUniversitydidastudyofcourseevaluationdata
from19982001.Usingthesameform,datawasanalyzedfrom29courses(20usingthepaperversion,9using
theonlineversion).Thestudyexaminedresponseratesandmeanscoresbetweenthemethods.Whilespecific
responseratesvaried,onlinewastypicallylowerthanthepaperform.Forexample,infall2000paperwas69%
comparedwith47%online.Usinga5pointscaleontheir13questions,4questionshadasignificantdifference
inmeanscoresbetweenmethods.Thiswasagreaterthan0.10differencewiththewebhavingthehighermean
score.Theother9questionshadalessthan0.10differenceinmeanscoresagainwithwebhavingthehigher
means.

Ballantyne,C.S.(2003).Onlineevaluationsofteaching:Anexaminationofcurrentpracticeand
considerationsforthefuture.InSorenson,D.L&Johnson,T.D(Eds)OnlineStudentRatingsof
Instruction,NewDirectionsforTeachingandLearning,No.96,Winter2003,JosseyBass

Thisarticlesummarizessomeoftheknownissuesrelatedtoonlinesurveys,usingMurdochUniversitys
implementationofanonlinecourseevaluationsystemasacasestudy.Responseratesareoftenlowerthan
desiredbutcanbeincreasedwithstrategiessuchasprovidingcomputeraccess,havingfacultysupportforthe
system,andlettingstudentsknowhowtheirfeedbackisused.Onlinesystemsneedtobedesignedtoprevent
multipleratingsofacoursebythesamestudentwhilestillprotectingstudentsanonymity.Quantitativeratings
aresimilartothosecompletedonpaper,whilecommentsaremoreplentifulandmorethoughtful.Onlinerating
systemsaresignificantlylessexpensivethanpaperevaluationsystems.

Ballantyne,C.S.(2004).Onlineoronpaper:Anexaminationofthedifferencesinresponseand
respondentstoasurveyadministeredintwomodes.PaperpresentedtotheAustralasian
EvaluationSocietyAnnualConference,Adelaide,SouthAustralia,1315October,2004.
http://www.aes.asn.au/conference2004/index.htm#fri

In2003MurdochUniversitycarriedoutasatisfactionsurveyofallstudents.Initialcontactwasviaemailasking
studentstorespondonline.Followupsofnonrespondentsusedthemoretraditionalmailout/paperformat.A
responserateoffiftypercentwasachievedwithsixtythreepercentofresponsescomingviatheonlinemode.
Malestudents,youngerstudents,undergraduatesandfulltimestudentsweremorelikelytorespondonline.
Studentsrespondingonlinewerelesslikelytocomment,butonlinecommentswerelengthierthanpaper
comments.

Bothell,T.W&Henderson,T.2003Doonlineratingsofinstructionmake$ense?InSorenson,
D.L&Johnson,T.D(Eds)OnlineStudentRatingsofInstruction,NewDirectionsforTeachingand
Learning,No.96,Winter2003,JosseyBass

Researcherscomparedthecostofanonlineevaluationsystemcomparedtopaperevaluations.Theyfoundthat
whenBrighamYoungUniversityswitchedtoonlineevaluations,itsavedthem$235,000ayear.Theestimated
costatBYUforpaperevaluationsis$1.06perstudentratingform,comparedwith$0.47peronlinestudent
ratingform.Thesavingscomefromareductioninprintingcosts,adecreasedneedforpersonnelhelpwith
collection,processing,andreporting,andfewertimetakenawayfrominstructorsintheclassroom.

BrighamYoungUniversityStudentratings:Frequentlyaskedquestions.Retrieved15thApril
2010fromhttps://studentratings.byu.edu/info/students/faq.asp

ProvidesinformationtostudentsabouttheBYUonlinecourseevaluations.Studentscannotratecoursesafter
finalexaminationsbegin.Studentsareassuredofanonymitybutgiventheoptiontoprovidetheirnametothe
instructorif,forexample,theinstructorsoffersextracreditforcompletingevaluations(thenameisnot
associatedwithresults,andisonlyvisibletotheinstructorifatleast5studentscompleteevaluationsforthat
class).Studentscanseetheresultsforonlyfouritems,whichareassociatedwithstudentlearning,andonlyif
theycompleteevaluationsforalltheirclasses.

Carini,R.M,Hayek,J.C.,Kuh,G.D.&Ouimet,J.A.(2003)."Collegestudentresponsestoweband
papersurveys:Doesmodematter?",ResearchinHigherEducation,2003,44,(1),P119
Retrieved13thSeptember2004fromhttp://www.kluweronline.com/issn/03610365/contents

Weexaminedtheresponsesof58,288collegestudentsto8scalesinvolving53itemsfromtheNationalSurvey
ofStudentEngagement(NSSE)togaugewhetherindividualsresponddifferentlytosurveysadministeredviathe
Webandpaper.Ourfindingssuggestthatmodeeffectsforfirstyearandseniorcollegestudentsgenerallytend
tobesmall.Anotableexceptioninvolvesitemsrelatedtocomputingandinformationtechnology,whichexhibit
morefavorableresponseswhenansweredviatheWeb.However,ourdatadonotallowustodiscernwhether
thisisatruemodeeffectorwhetherthosemostengagedincomputingandinformationtechnologyarealso
thosewhogravitatetowardtheWebbasedmodes.

Cates,W.M.(1993).Asmallscalecomparisonoftheequivalenceofpaperandpenciland
computerizedversionsofstudentendofcourseevaluations.ComputersinHumanBehavior,9,
401409.
Thisstudycomparedresponsestotwoversionsofanendofcourseevaluationinstrumentcompletedby
graduatestudents:thetraditionalprintedformcompletedusingpencilandpaper,andamicrocomputerbased
formthatpresentedequivalentitemsandacceptedstudentresponses.Afindingofnosignificantdifferencein
favorablenessofcompositeratingsbetweenthetwoversionspromptedtheresearchertoperformitembyitem
analysesofthetwoinstruments.Theseanalysesrevealedthatratingsoftheindividualitemsononeinstrument
werehighlycorrelatedwiththeratingsoftheirmatchedcorrespondingitemsontheotherinstrument.The
paperandpencilandcomputerizedevaluationinstrumentswerefoundtobeofalmostidenticallyhigh
reliability.

Chen,Y.&Hoshower,L.B.(2003)."Studentevaluationofteachingeffectiveness:anassessment
ofstudentperceptionandmotivation."Assessment&EvaluationinHigherEducation28,(1)
7288.
Overthepastcentury,studentratingshavesteadilycontinuedtotakeprecedenceinfacultyevaluationsystems
inNorthAmericaandAustralia,areincreasinglyreportedinAsiaandEuropeandareattractingconsiderable
attentionintheFarEast.Sincestudentratingsarethemost,ifnottheonly,influentialmeasureofteaching
effectiveness,activeparticipationbyandmeaningfulinputfromstudentscanbecriticalinthesuccessofsuch
teachingevaluationsystems.Nevertheless,veryfewstudieshavelookedintostudents'perceptionofthe
teachingevaluationsystemsandtheirmotivationtoparticipate.Thisstudyemploysexpectancytheoryto
evaluatesomekeyfactorsthatmotivatestudentstoparticipateintheteachingevaluationprocess.Theresults
showthatstudentsgenerallyconsideranimprovementinteachingtobethemostattractiveoutcomeofa
teachingevaluationsystem.Thesecondmostattractiveoutcomewasusingteachingevaluationstoimprove
coursecontentandformat.Usingteachingevaluationsforaprofessor'stenure,promotionandsalaryrise
decisionsandmakingtheresultsofevaluationsavailableforstudents'decisionsoncourseandinstructor

selectionwerelessimportantfromthestudents'standpoint.Students'motivationtoparticipateinteaching
evaluationsisalsoimpactedsignificantlybytheirexpectationthattheywillbeabletoprovidemeaningful
feedback.Sincequalitystudentinputisanessentialantecedentofmeaningfulstudentevaluationsofteaching
effectiveness,theresultsofthisstudyshouldbeconsideredthoughtfullyastheevaluationsystemisdesigned,
implemented,andoperated.

Collings,D.&Ballantyne,C.S.(2004,November2425).Onlinestudentsurveycomments:A
qualitativeimprovement?Paperpresentedatthe2004EvaluationForum,Melbourne,Victoria.
http://www.tlc.murdoch.edu.au/pubs/docs/Eval_forum_paper.pdf
Giventhatonlineevaluationstendtohaveadecreaseinresponseratebutanincreaseincommentscompared
topaperevaluations,theseresearchersquestionedwhetheranincreasedresponserateonlinewouldleadto
lessvaluablecomments.Thiswouldbelikelyifthestudentsmosteagertoparticipatewerealsomostlikelyto
comment.However,theyfoundthatregardlessofwhenstudentsrespondedtothesurvey,thepercent
commentingandthelengthofcommentswerenearlythesame,withaslightdecreaseforthoseresponding
neartheendofthetimeperiod.Theysuggestthatqualitativefeedbackmaybemorevaluablethanquantitative
feedback,andincreasingresponseratesisntnecessaryforqualityfeedback.

Cummings,R.andBallatyne,C.(1999).Studentfeedbackonteaching:Online!Ontarget?
PaperpresentedattheAustralisianSocietyAnnualConference,October,1999.

MurdochUniversitySchoolofEngineeringranapilotin1999ofonlinecourseevaluationsusingthesameform
onlineasonpaper.Studentsfoundtheonlineformeasier,faster,andfeltitofferedgreateranonymity.The
schoolhasa50%mandateforresponserateincourseevaluations.Typicallypaperevaluationshada65%
responserate.Theonlinepilotaveraged31%with4ofthe18coursesoverthe50%mandate.Theresponse
raterangewasawide3%to100%.Becausethepilotwasinadequatelypromoted,somefacultydidntknow
theywereusingonlineformsanddidntadequatelypreparestudents.Studentsnotedthattheyfeltnopressure
tofillouttheonlineevaluations.Theinvestigatorsconcludedthatthequalityofresponseswasthesame
becausetheyreceivedthesameamountofcommentsonline,whichiswhatisusedmostfromtheevaluation
form.

Dommeyer,CJ.,Baum,P.,Chapman,KS.,andHanna,RW.(2003).Anexperimental
investigationofstudentresponseratestofacultyevaluations:Theeffectoftheonlinemethod
andonlinetreatments.PaperpresentedatDecisionSciencesInstitute;Nov.2225,2003;
Washington,DC.

TheCollegeofBusinessAndEconomicsatCaliforniaStateUniversity,Northridgedidastudywith16professors
toseehowthemethodofevaluationaffectsresponserateandifonlinetreatments(incentives)affectthe
responserate.Eachprofessortaught2sectionsofthesameundergraduatebusinesscourse.Thesameformwas
usedinbothmethods.Instructorswererandomlyassignedinto1of4groupsusingdifferentincentives:0.25%
gradeincentiveforcompletionofanonlineevaluation(4courses),inclassdemonstrationonhowtodothe
onlineevaluation(2courses),if2/3oftheclasssubmittedonlineevaluationsstudentswouldreceivetheir
finalgradesearly(2courses),oracontrolgroup(8courses).Theonlineevaluationsaverageda43%response
rateandthepaperevaluationsaveraged75%.Lookingatjustthecontrolgroup,theiraverageresponseratewas
29%.Intheindividualcasestheincentiveshadtheeffectofincreasingresponserate(gradeincentive87%
responserate,demonstration53%,andearlyfinalgrade51%).

10

Dommeyer,C.J.,Baum,P.Hanna,R.W.,&Chapman,K.S.(2004)"Gatheringfacultyteaching
evaluationsbyinclassandonlinesurveys:theireffectsonresponseratesandevaluations"
Assessment&EvaluationinHigherEducation29,(5)611623.
Thisstudycomparesstudentevaluationsoffacultyteachingthatwerecompletedinclasswiththosecollected
online.Thetwomethodsofevaluationwerecomparedonresponseratesandonevaluationscores.Inaddition,
thisstudyinvestigateswhethertreatmentsorincentivescanaffecttheresponsetoonlineevaluations.Itwas
foundthattheresponseratetotheonlinesurveywasgenerallylowerthanthattotheinclasssurvey.
Additionally,thestudyfoundthatonlineevaluationsdonotproducesignificantlydifferentmeanevaluation
scoresthantraditionalinclassevaluations,evenwhendifferentincentivesareofferedtostudentswhoare
askedtocompleteonlineevaluations.

Donovan,J.,Mader,C.,andShinsky.J.,(2006)Constructivestudentfeedback:Onlinevs.
traditionalcourseevaluations.JournalofInteractiveOnlineLearning.Volume5,Number3,
Winter2006.

Abstract:Substantialeffortshavebeenmaderecentlytocomparetheeffectivenessoftraditionalcourseformats
toalternativeformats(mostoften,onlinedeliverycomparedtotraditionalonsitedelivery).Thisstudy
examines,notthedeliveryformatbutrathertheevaluationformat.Itcomparestraditionalpaperandpencil
methodsforcourseevaluationwithelectronicmethods.Eleveninstructorstookpartinthestudy.Each
instructortaughttwosectionsofthesamecourse;attheend,onecoursereceivedanonlinecourseevaluation,
theotheratraditionalpencilandpaperevaluation.Enrollmentinthese22sectionswas519students.
Researchersanalyzedopenendedcommentsaswellasquantitativerankingsforthecourseevaluations.
Researchersfoundnosignificantdifferencesinnumericalrankingsbetweenthetwoevaluationformats.
However,differenceswerefoundinnumberandlengthofcomments,theratioofpositivetonegative
comments,andtheratioofformativetosummativecomments.Studentscompletingfacultyevaluationsonline
wrotemorecomments,andthecommentsweremoreoftenformative(definedasacommentthatgave
specificreasonsforjudgmentsothattheinstructorknewwhatthestudentwassuggestingbekeptor
changed)innature.

Emery,L.,Head,T.,Zeckoski,A.,andYuBorkowski,E.(2008)DeployinganOpenSource,
OnlineEvaluationSystem:MultipleExperiences.PresentationatEducause2008,October31,
Orlando,FL.

Fourinstitutions,UniversityofMichiganAnnArbor,VirginiaTech,UniversityofCambridgeandUniversityof
Maryland,collaboratedonanopensourceonlineevaluationsystemwithinSakai.Responseratesinthevarious
pilotsrangedfrom32%to79%.Theyfoundthekeybenefitsofonlineevaluationstobesecurity,validity,
efficiency,costsavings,rapidresultsturnaroundandhigherqualitystudentcomments.

Ernst,D.(2006)StudentEvaluations:AComparisonofOnlinevs.PaperDataCollection.
PresentationatEducause2006,October10,Dallas,TX.

TheCollegeofEducationandHumanDevelopmentattheUniversityofMinnesotadidastudyon314classpairs
(14,154studentevaluations)fromfall2002tofall2004.Thegoalsweretoseeifthereisadifferenceinresponse
rate,adifferenceinresponsedistributions,adifferenceinaverageratingsbetweenthetwomethodsandwhat
arethecommonperceptionsofeachmethod.Inthestudygrouptheonlineformaverageda56%response
ratewhereasthepaperversionaveraged77%.Slightlymorestudentsrespondedonthehighandlowendsof
the7pointscalethandidinthemiddle.Therewasnosignificantdifferenceinthemeanratingon4required
questions.

11

eXploranceInc.,AFreshLookatResponseRates.WhitePaper.
http://www.explorance.com/Education/brochures/A%20Fresh%20Look%20at%20Response%2
0Rates.pdf

Thiswhitepaperoutlines9bestpracticesformovingtoonlinecourseevaluations.Keybenefitstomoving
onlinearelistedaswellasstrategiestobuildresponserates.

Fraze,S.,Hardin,K.,Brashears,T.,Smith,J.,Lockaby,J.(2002)TheEffectsOfDeliveryMode
UponSurveyResponseRateAndPerceivedAttitudesOfTexasAgriScienceTeachers.Paper
presentedattheNationalAgriculturalEducationResearchConference,December1113,Las
Vegas,NV,

TexasTechUniversitystudied3modesofsurveyingarandomgroupofTexasAgriScienceteachers.The3modes
wereemail,web,andpaper.Nosignificantdifferenceinthereliabilityoftheresponseswasfound.However,
theresponserateswere60%,43%and27%forpaper,webandemailrespectively.

Handwerk,P.,Carson,C.,andBlackwell,K.(2000).Onlinevs.paperandpencilsurveyingof
students:Acasestudy.Paperpresentedatthe40thAnnualMeetingoftheAssociationof
InstitutionalResearch,May2000(ERICdocumentED446512).

TheUniversityofNorthCarolinaatGreensborodidastudyofusingandonlineversionofafeedbacksurveyfor
determiningwhystudentsselectedordidnotselectGreensboro.Theyfoundtheonlineversiongeneratedmore
commentsthoughhadalower(26%)responseratethanthepaperversion(33%).Nosignificantdifferencewas
foundintheresponsecontentbetweenthetwomethods.

Hardy,N.2003Onlineratings:Factandfiction.InSorenson,D.L&Johnson,T.D(Eds)Online
StudentRatingsofInstruction,NewDirectionsforTeachingandLearning,No.96,Winter2003,
JosseyBass

ThisstudyuseddatafromNorthwesternUniversitysimplementationofanonlineevaluationsystemtorefute
mythssurroundingonlinecourseevaluations.Contrarytothefearsofsomefacultymembers,onlineratings
werenotmorelikelythanpaperevaluationstoproducenegativeratingsorcomments,andstudentswrote
substantiallymorecommentsontheonlineevaluations.Additionally,anygivenclassmayhaveahigher,lower,
orsimilarresponseratewhenswitchingfrompapertoonline.

Hmieleski,K.andChampagne,M.2000"Pluggingintocourseevaluation."Assessment,
September/October2000.

TheIDEALaboratorysurveyedthenation's200mostwiredcollegesasidentifiedbyZDNet.Surprisingly,98%of
the"mostwired"schoolsuseprimarilypaperbasedevaluationforms.Oftheschoolsrequiringsomeformof
courseorfacultyevaluation,allcurrentlyadministertheevaluationformssolelyattheendoftheterm(the
"autopsyapproach").Sixtysevenpercentofschoolsreportedreturnratesof70%orhigherforpaperbased
evaluation.SchoolsusingorpilottestingaWebbasedevaluationsystemreportedreturnratesrangingfrom
20%togreaterthan90%.Only28%ofrespondentsratedtheirfacultyasverysupportiveoftheirschool's
currentevaluationsystem.Ninetyfivepercentofschoolsreportedthattheirfacultymembersareinvolvedin
thedevelopmentofcourseevaluations,typicallythroughparticipationinthefacultysenateorbydeveloping
evaluationquestions.Thirtyonepercentofschoolsreportedthatstudentsareinvolvedinthedevelopmentof
theircollegescourseevaluationsystem,typicallythroughparticipationinthestudentsenate,and36%of
schoolsallowtheirstudentstoviewtheresultsofcourseevaluations,typicallyviatheInternetandstudent

12

publications.Theysuggestafeedbackandrefinementbywhichstudentscanprovidefeedbackthroughout
thecoursetoallowinstructorstomakerapidchangestothecourse,andfoundthatwhenusingafeedbackand
refinementsystem,commentstendtobemoreplentifulandinsightful.Additionally,theynotethatwhen
responsesarerequired,responseratesapproach100%butvaluablecommentsdropdramatically.

Hoffman,K.M.2003Onlinestudentratings:Willstudentsrespond?InSorenson,D.L&
Johnson,T.D(Eds)OnlineStudentRatingsofInstruction,NewDirectionsforTeachingand
Learning,No.96,Winter2003,JosseyBass

ThisinvestigationwasintendedasanupdatetoHmielskiandChampagne(2000)sarticleoncollegesusingpaper
oronlineevaluationforms.Oftheinstitutionssurveyed,90%werestillusingaprimarilypaperbased
evaluationprocess,and12%wereusingnonscannablepaperforms.However,56%wereusingtheInternetfor
theevaluationofonlinecoursesorwereplanningtoimplementanonlineratingssystemforonlinecoursesin
2003.MoreschoolsusedtheInternettoreportevaluationresultstofacultythanusedtheInternettocollect
ratingsfromstudents;additionally,12%ofinstitutionsallowedstudentstoviewevaluationresults.

Johnson,T.D.2003Onlinestudentratings:Willstudentsrespond?InSorenson,D.L&
Johnson,T.D(Eds)OnlineStudentRatingsofInstruction,NewDirectionsforTeachingand
Learning,No.96,Winter2003,JosseyBass

BrighamYoungUniversityexperimentedwithdifferentstrategiesforincreasingresponseratestoonlinecourse
evaluations.Wheninstructorsassignedstudentstocompletecourseevaluations,whetherornottheygave
pointsfortheassignment,therewasalargejumpinresponserates.Additionally,whentheevaluationform
wasshort,studentstookthetimetowritemoreopenendedcomments.Studentswhodidnotrespondmost
oftendidnotknowabouttheonlineevaluations.Infocusgroups,studentstopsuggestionforincreasing
responserateswastoallowearlyaccesstogradesforthosewhocompletedtheevaluations.

Kasiar,J.B.,Schroeder,S.L.&Holstad,S.G.(2002).Comparisonoftraditionalandwebbased
courseevaluationprocessesinarequired,teamtaughtpharmacotherapycourse.American
JournalofPharmaceuticalEducation,66(3),26870.
Inateamtaughtcourse(enrollment=169),studentswererandomlyassignedtocompleteevaluationsonline(n
=50)orbytraditional,paperbasedmethods(n=119).Webbasedandtraditionalevaluationswerecompared
forLikertscore,quantityandqualityofstudentcomments,studentsatisfaction,andconsumptionofstaffand
facultytime.Of252questionsaskedofeachstudent,72(29percent)hadasignificantlydifferentLikertscore.In
allbuttwoquestions,however,themedianand/orrangewasdifferentbyonlyonepointandinmostcasesdid
notchangetheoverallmeaningoftheresponse(e.g.,amedianresponseofStronglyAgreeratherthan
Agree.)Thenumberofcommentswassignificantlyhigherinthewebbasedgroupcomparedtothe
traditionalgroup.Students,facultyandstaffallratedthewebprocessasmoreconvenientandlesstime
consumingthanthetraditionalmethod.Awebbasedevaluationsystemusingsubsetsofstudentstocomplete
eachevaluationcanbeemployedtoobtainrepresentativefeedback.Thewebbasedprocessyields
quantitativelyandqualitativelysuperiorstudentcomments,enhancedstudentsatisfaction,andmoreefficient
useoffacultyandstafftime.

13

Laubsch,P.(2006).Onlineandinpersonevaluations:Aliteraturereviewandexploratory
comparison.JournalofOnlineLearningandTeaching,2(2).
http://jolt.merlot.org/Vol2_No2_Laubsch.htm

TheMasterofAdministrativeScienceprogramatFairleighDickinsonUniversityperformedastudyoncourses
taughtbyadjunctfaculty.Theonlineevaluationsreceiveda61%responserateandtheinclassevaluations
receiveda82.1%responserate.Theyfoundthattheonlineevaluationsreceivedtwiceasmanycomments
(countingtotalwords)astheinclassevaluations.Onthequestionaboutmaterialsbeingclearlypresented
(focusedonthefacultymember)thevariationinmeanscoresinonlineandinclasswas0.33ona5pointscale
withonlinehavingalesspositiverating.Thisisastatisticallysignificantdifference.Administratorsnotedthat
bothmeanswerebetterthantheagreeandwerenotconsideredpoorratings.

LayneB.H.,DeCristoforJ.R.,McGintyD(1999).Electronicversustraditionalstudentratingsof
instruction.ResHigherEduc.1999;40:22132.

Atasoutheasternuniversity66coursesmadeupof2453studentsdidacomparisonofresponseeffectsbetween
paperandpencilandonlineusingthesameform.Halfdidonlineandhalfdidpaperandpencilforms.The
onlineresponseratewas47%andthetraditionalgroupwas60%.Also,76%oftheonlineformsprovided
commentscomparedto50%ofthetraditionalforms.Nosignificantdifferencewasfoundinmethods.

Liegle,JOandDSMcDonald.LessonsLearnedFromOnlinevs.PaperbasedComputer
InformationStudents'EvaluationSystem.InTheProceedingsoftheInformationSystems
EducationConference2004,v21(Newport):2214.ISSN:15427382.(Alaterversionappearsin
InformationSystemsEducationJournal3(37).ISSN:1545679X.)
http://proc.isecon.org/2004/2214/index.html
GeorgiaStateUniversityCollegeofBusinessranavoluntarypilotfrom2002to2003usinganidenticalonline
versionoftheirpapercourseevaluationformintheDepartmentofComputerInformationSystems.Faculty
fearedanonlineformwouldyieldlowerscoresandlowerresponserates.Inparticular,thefearwasthatfew
studentswouldsubmitonlineevaluations,poorstudentswouldtakerevengeonthefacultyandgoodstudents
wouldntbother.Thepaperformhada67%responserateandtheonlineformhadan82%responserate.This
likelyduetothefactthattheCISdepartmenthadeasyaccesstocomputerlabsforstudentstotakethe
evaluationsonline.Usingaquestiononteachereffectiveness,thestudyfoundnosignificantdifference
betweenthemethods.Goodstudentsparticipatedinthesamenumbersandweakerstudentsdidfewer
onlineevaluations.

Lovric,M.(2006).Traditionalandwebbasedcourseevaluationscomparisonoftheirresponse
ratesandefficiency.Paperpresentedat1stBalkanSummerSchoolonSurveyMethodology.
http://www.balkanprojectoffice.scb.se/Paper%20Miodrag%20Lovrich_University%20of%20Belg
rade.pdf
Thepaperpresentsashortliteraturereviewcomparingonlineevaluationswithpaper.TheEconomics
departmentatUniversityofBelgrade,Serbiaconductedasmallpilotinacourseof800studentsinMayof2006.
Halfthestudentsreceivedpaperevaluationsinclassandhalfweredirectedtocompleteanidenticalonline
evaluation.Thepaperevaluationreceiveda92.5%responserateandtheonlinereceiveda52%responserate
afteranincentivewasintroduced.Theyfoundthatnearlytwiceasmanystudentsfilledouttheopenended
questiononlinewhencomparedtothepapergroup.Ontheinstructorrelatedquestionstheyfoundavariation
of0.09to0.22ona10pointscale.Nostatisticalanalysiswasdoneforsignificance.

14

Matz,C.(1999).Administrationofwebversuspapersurveys:Modeeffectsandresponse
rates.MastersResearchPaper,UniversityofNorthCarolinaatChapelHill.(ERICdocument
ED439694).

InasurveyofacademicreferencelibrariansinNorthCarolina,Matzfoundnosignificantdifferenceinresponse
contentsbetweenthemethodsused.Theonlineformhada33%responserateandthepaperformhada43%
responserate.

Monsen,S.,Woo,W.,Mahan,C.Miller,G.&W(2005).OnlineCourseEvaluations:Lessons
Learned.PresentationatTheCALIConferenceforLawSchoolComputing2005.

YaleLawstartedonlinecourseevaluationsin2001withalessthan20%responserate.Thecurrent8question
formisrunbystudentrepresentativesandhasa90%responserate.Studentscannotseetheirgradesuntil
theyfillouttheevaluation.NorthwesternUniversitySchoolofLawstartedonlinecourseevaluationsin2004.
Sofartheyhavea68%responseratewhichcomparestoa7080%paperresponserate.Northwesternisagainst
usinganypenalties(withholdinginformationfromastudentuntiltheyfilloutanevaluation).TheUniversityof
DenverSturmCollegestartedonlinecourseevaluationsin2002withapilotof10courses.Thepilothadan83%
responserate.Continuinginto2003thepilotexpandedto80courses(withan81%responserate)andthen
expandedtoalloftheirofferings(witha64%responserate).Currentlytheymaintainaresponseratearound
70%.DukeLawstartedonlinecourseevaluationsin2003whentheirscantronmachinebrokeandtheexpense
ofreplacingwastoogreat.Theyproposedagoalof70%responserateandusedthesameformonline.Thefirst
termaverageda66%responserate(with29%ofthe82coursesreachingthe70%goal).Inspring2004the
averagewas60%(with30%ofthe119coursesreachingthe70%goal).Infall2004theaveragewas52%(with
8%ofthe93coursesreachingthe70%goal).Inspring2005,afterdroppingnonlawstudentsfromthepool,
theaveragewas67%(with41%ofthe117coursesreachingthe70%goal).Theschoolisconsideringseveral
penaltiesforfailuretofilloutanevaluationwithholdingregistration,withholdinggrades,orwithholdingfree
printing.

Norris,J.,Conn,C.(2005).Investigatingstrategiesforincreasingstudentresponseratestoon
linedeliveredcourseevaluations.QuarterlyReviewofDistanceEducation2005;6(1)p1332
(ProQuestdocumentID975834871).

Thispaperreportsthefindingsof2studiesdoneatNorthernArizonaStateUniversity.Thefirststudylookedat
historicdatafrom20002002toexaminestudentresponsestoonlinecourseevaluationsin1108course
sections.Thisgrouphadanaverageresponserateof31%.Afollowupquestionnairewassentto50facultyin
thegrouptoexplorewhatstrategiesimprovedresponserate.Theseresultsinformedthesecondstudyon39
onlinecoursesectionsand21sectionsofarequiredfreshmanfacetofacecourse.Thesecondstudyusedsome
basicstrategies(nopenaltystrategies)intheimplementationoftheonlinecourseevaluations:2weeksbefore
theendofthecoursetheURLtoevaluationwaspostedinthecoursemanagementsystem,anannouncement
containingastatementofcourseevaluationvalueandduedatewassentinamethodappropriatetotheclass
(email,onlinesyllabusordiscussionboard),andareminderemailwassent1weekbeforetheclassended
containingtheURLandduedate.The39onlinecoursesectionsaverageda74%responserateandthe21face
tofacecoursesaverageda67%responserate.Inaddition,11sectionsofthefacetofacecourseusedpaper
evaluationsandreceiveda83%responserate.Thesesuggestionsareverysimilartotheemergingfindingsfrom
theTLTGroupsBeTAproject.

15

OfficeofInstitutionalResearchandAssessment,MarquetteUniversity,OnlineCourse
EvaluationPilotProjectatMarqetteUniversity.Spring2008.
http://www.marquette.edu/oira/ceval/

MarquetteUniversitymovedfromacopyrightedinstrument,IAS,totheirowninstrument,MOCES.Becauseof
thecopyrightconcernsthenewinstrumenthasrewordeditemsthatmaintaintheintentoftheIASitems.In
springsemesterof2008apilotwasconductedin124coursesectionswith3837students.Theyevaluatedthe
effectivenessofanonlineapproachversuspaperandpencilandthesoftwareusedtodelivertheevaluations.
Responseratesonlinewerelowerin3ofthe5pilotdepartments,comparablein1andhigherin1when
comparedto3semesteraveragesofpaperandpencilforms.Apoweranalysisoftheresponseratesrevealed
therateswerehighenoughof95%confidenceintheresults.Therewasnosignificantdifferenceinthemean
ratingsforthe4corequestionsbetweentheoldIASformandtheMOCESonlineform.

OnlineCTEProjectTeam.(2005).Onlinecourseandteachingevaluation:Reportonatrialrun
withrecommendations.TeachingandLearningCenter,LingnanUniversity.
http://www.ln.edu.hk/tlc/level2/pdf/online%20cte%20report%20050411.pdf
Apilotof18classesusedanonlinecourseandteachingevaluation(CTE)atLingnanUniversity.Formostclasses,
amemberoftheprojectteamwenttoascheduledclassduringtheevaluationperiodandexplainedtostudents
thenatureandpurposeoftheonlineCTEtrial.Theaverageresponserateforthe18classeswas69.7%.Classes
withlowresponseratescorrespondedtothosethathadalargenumberofundeliverableemailorhadnot
receivedabriefingfromaprojectteammember.Nosignificantdifferenceswerefoundinmeanscores
betweenonlineevaluationsandpreviouspaperevaluationsforthesameinstructorandcourse.Only3CTEs
recordedmorecommentsbystudentsthaninthepreviouspaperbasedCTES;however,theonlineCTEs
containedmoreelaboratecomments.Studentfeedbackindicatedthatstudentsgenerallypreferredtheonline
CTE;concernswereprimarilyabouttheanonymityoftheirresponsesbecausetheywererequiredtologinto
theevaluationsystem.

Sax,L.,Gilmartin,S.,Keup,J.,Bryant,A.,andPlecha,M.(2002).Findingsfromthe2001pilot
administrationofYourFirstCollegeYear(YFCY):Nationalnorms.HigherEducationResearch
Institute,UniversityofCalifornia.

TheYFCYdistributeditssurveythatassessesstudentdevelopmentduringthefirstyearincollegeusing3
methods:online,onlineorpaper,andpaper.Inapoolof57schools,16usedthealternativemethodsof
distribution.Thestudyfoundnosignificantdifferenceinresponsesbetweenthemethods.Theresponserate
overallwas21%.Theonlineonlymethodresponseratewas17%andtheonlineorpapergrouphada24%
responserate.

Schawitch,M.(2005)OnlineCourseEvaluations:OneInstitutesSuccessinTransitioningfrom
aPaperProcesstoaCompletelyElectronicProcess!PresentationattheAssociationfor
InstitutionalResearchForum,June2005.

TheRoseHulmanInstituteofTechnologypilotedanonlinecourseevaluationin2002withasmallgroupof
faculty.Overtheacademicyearthepilothada70%responserate.77%ofstudentspreferredtheonlinemode
andfacultyreactedpositivelytothepilot.In2003theentirecampusadoptedtheonlineform.Overthe3
terms,theonlineevaluationshadresponseratesof86%,78%and67%.In2004the3termshad75%,71%and
67%.Historicallypaperevaluationshadan8587%responserate.Theyareinvestigatingvariousincentive
possibilities.

16

Spencer,K.&Schmelkin,L.P.(2002)"StudentPerspectivesonTeachinganditsEvaluation."
Assessment&EvaluationinhigherEducation,27(5)397409.
Theresearchonstudentratingsofinstruction,whilevoluminous,hashadminimalfocusontheperceptionsof
thestudentswhodotheratings.Thecurrentstudyexploredstudentperspectivesoncourseandteacherratings
aswellassomeissuesrelatedtoteachingeffectivenessandfacultyroles.Itwasfoundthatstudentsare
generallywillingtodoevaluationsandtoprovidefeedback,andhavenoparticularfearofrepercussions.

Thorpe,S.W.(2002).Onlinestudentevaluationofinstruction:Aninvestigationofnonresponse
bias.Paperpresentedatthe42ndAnnualForumoftheAssociationforInstitutionalResearchin
TorontoCanada.Retrieved9thNovember2004,fromhttp://www.airweb.org/forum02/550.pdf

DrexelUniversitystudiedwhethersignificantdifferencesexistinstudentresponsestocourseevaluationsgiven
onpaperandonlinein3courses.Responseratesinthe3classesforpaperandonline(respectively)were37%
and45%,44%and50%,70%and37%.Incomparingstudentswhorespondedtotheevaluationsacrossthe3
coursesthestudyfoundthatwomenweremorelikelythanmentorespond,studentswhoearnedhigher
gradesweremorelikelytorespond,andstudentswithahigheroverallGPAweremorelikelytorespond.For
twocoursestheonlineevaluationshadaslightlyhigheraverageitemrating.Fortheothercourse2significant
differenceswerefound:studentsdoingtheonlineevaluationwerelesslikelytoparticipateactivelyand
contributethoughtfullyduringclassandtoattendclasswhencomparedtothepaperevaluationgroup.Butthe
responsesoverallwerenotsignificantlydifferent.

17