Runninghead:MEETINGTHELLCOUTCOMES

MeetingtheLeadershipLearningCommunitysOutcomes:
AnEvaluationPlan
DeNeatriaRobinson&EricRuelle
LoyolaUniversityChicago
ELPS431

MEETINGTHELLCOUTCOMES

TableofContents
Introduction

ProgramContext&History

Evaluation&Significance

PurposeoftheEvaluation

Stakeholders

LogicModel

10

QuantitativeApproach

12

Population&SamplingFrame

12

ResearchDesign&Method

13

SurveyInstrument

15

Statistical&DataAnalysis

17

Validity&Reliability

18

QuantitativeDataPresentation

18

QualitativeApproach

19

FocusGroupParticipants

20

FocusGroupProtocol

21

FocusGroupImplementation

22

AnalysisofData

23

CodingProcess

24

Validity&Reliability

24

Limitations

25

DataPresentation

26

Budget

27

NextSteps

27

References

28

Appendices

MEETINGTHELLCOUTCOMES

AppendixA:LCCLearningOutcomes

29

AppendixB:LogicModel

31

AppendixC:PretestSurvey

32

AppendixD:PosttestSurvey

35

AppendixE:FocusGroupProtocol

38

AppendixF:FocusGroupConsentForm

40

AppendixG:Timeline

41

AppendixH:Budget

42

AppendixI:PosterPresentation

43

MEETINGTHELLCOUTCOMES

TheLeadershipforSocialChangeLearningCommunityatLoyolaUniversityChicago
focusesontheleadershipdevelopmentofincomingfirstyearundergraduatestudentswhoare
seekingtolearnmoreabouttheirownleadershipstylesandhaveapositiveimpactontheir
community.AsoneofsevenoncampusLearningCommunities(LCs),theLeadershipLC
(LLC)connectsstudentsacademicandresidentialexperiencestofosteraclosersenseof
community,bothwiththeuniversityandresidencehall,aswellasengagestudentsinanareaof
personalinterest.Studentsintheprogramtakecoursesthatfulfilldegreerequirementsrelatedto
leadership.TheythencontinuethosediscussionsonthefirstfloorofSimpsonHall,onLoyolas
LakeshoreCampus,whereallofthemlive.Programsareofferedthroughthedepartmentsof
ResidenceLifeandStudentLeadershipDevelopment,thatarecosponsorsfortheLLC,inorder
tosupportstudentsthroughouttheyear.Thisproposalisamixedmethodsapproachat
examiningtheextenttowhichfirstyearstudentsintheLLCareeffectivelymeetinglearning
outcomesandacquiringspecificcompetenciesrelatedtosocialchangeandleadership.Itis
separatedintopartsthatwillbreakdowntheevaluationplansuchasthecontextoftheprogram,
detailsofthequantitativeandqualitativeplans,andthewaysinwhichthedatawillbe
interpretedandpresented.
ProgramContext&History

TheResidenceLifeandStudentLeadershipDevelopment(SLD)partnershipwasintegral

intheLeadershipLearningCommunitysformation.OriginallyconceivedthroughResidence
Life,SLDwasbroughtontoprovidemoreintentionalitytothecocurricularportionoftheLLC.
Theprogramisnowinitsfourthyear,whichmeansthatthefirstyearstudentswhostartedwith
theprogramarenowseniorsthatareabouttograduate.Therehasbeensignificantchangeoverin

MEETINGTHELLCOUTCOMES

theleadershipinResidenceLifeoverseeingtheLLCduringthelastfouryears.MarciWalton,
AssistantDirectorforAcademicSupportandLearningCommunities,iscurrentlyproviding
supportfromResidenceLifewhereasShannonHowes,DirectorofStudentLeadership
Development,hasremainedaconstantrepresentationfromSLDsincetheinceptionoftheLLC.
Providingadditionalsupportaretwoacademicadvisorswhoworkwithstudentstomakesure
theircourseworkiscompletedandtrackedthroughtheirtimeatLoyola,aswellasaLearning
CommunityAssistant(LCA)thatispartiallyresponsibleforprogrammingwithintheLLC.
PleaserefertotheStakeholderssectionforfurtherinformationontheseparties.

ParticipationintheLLChasfluctuatedoverthelastfewyearsnumbershaveranged

fromfiftyfourtoonehundredstudentsintheprogram.Thecapacityislimitedtotheamountof
spacethatisavailableintheresidencehallfloorbuttheLLCanticipatesacceptingroughly
onehundredstudentsforthe20152016academicyear(LeadershipLC,2015).Studentsare
selectedonafirstcome,firstservebasis,andonceaccepted,theyarerequiredtotakeaFirst
YearSeminarclasstogether(UNIV101)intheirfirstsemester,whichservesasanintroduction
toLoyola(mission/howtosucceed).Studentsalsohavetheoptionofthreeelectivecourses
Ethics,PhilosophyandPersons,orSocietyinaGlobalAgewhichchallengestudentsto
investigatetheirvaluesandunderstandtheimpacttheyhaveonaglobalsociety.Classesare
cappedattwentythreetotwentyfourstudents.Ifstudentsfailtomeettherequirementslaidout
byResidenceLife,StudentLeadershipDevelopment,ortheiracademicsfallbelowacertain
threshold,theymeetwiththeAssistantDirectorforAcademicSupportandLearning,the
DirectorofStudentLeadership,ortheiracademicadvisorstodeterminewhethertheyshould
continueintheLLC.

MEETINGTHELLCOUTCOMES

20152016isalandmarkyearfortheLeadershipLC.AtthesametimethattheLLCis

welcomingtheirnewcohort,anewLeadershipminorwillstartupintheSpringsemesterthat
willgivestudentsmoreacademicoptionstochoosefrom.JohnDugan,AssociateProfessorin
LoyolasHigherEducationdepartment,createdthecurriculumforthenewLeadershipminor
th
thatmostLLCstudentswilltakepartin.ShannonHowes(personalcommunication,Sept.16
,

2015)reportedthatfocusgroupswereheldonthecocurricularportionandhaveyieldedhelpful
feedbackbutnothingthathasdramaticallyaffectedtheoperationoftheLLC.Similarly,
ResidenceLifehasdonesmallsurveysandrevealedthatstudentswhoparticipateinLCs
oncampushaveahigherGPAhowever,theremightbeaselfcollectionbias.Thisevaluation
plancomesataveryopportunetimetobridgethegapincurrentevaluationpracticeandtogive
detailedfeedbackontheeffectivenesstoreachthelearningoutcomesoftheLLCforstudents.
EvaluationSignificance
InadditiontothisacademicyearmarkingthefourthyearoftheLeadershipforSocial
ChangeLearningCommunity,thisyearalsomarkstherollingoutofthenewstrategicplanfor
theDivisionforStudentDevelopment,underwhichResidenceLifeishoused.Itisimportantto
ensurethattheLLCprogramisinlinewiththeinstitutionalvisionandmission.More
specifically,institutionalprioritynumbertwo,reimaginingknowledgetosolvecomplexsocietal
challenges(DivisionofStudentDevelopment,2015).
WithResidenceLifeexperiencingvarioustransitionthatcanleadtothecreation,
continuation,orremovalofspecificlearningcommunities,itisimportanttodocumentdatathat
supportstheimpactofsuchcommunitiesonthestudentexperienceandachievementoflearning
outcomes.TheLLCisthemostpreferredandhighestattendedlearningcommunitywithmany

MEETINGTHELLCOUTCOMES

membersapplyingforstudentleadershippositionswithintheResidenceLifeDepartment,
followingtheiryearinthelearningcommunity.Surveysandassessmentsduringthepasttwoto
threeyearshaveprimaryaskedLLCmembersaboutpreferredengagementopportunities,
communitybuilding,andawarenessofresources.Ourassessmentisdesignedtoevaluatea
missingbutkeycomponentwithintheseassessments:competencyinlearningoutcomes
particularly,learningoutcomesthatarespecifictotheLLC.
PurposeoftheEvaluation
ThepurposeoftheevaluationistoexaminetheextenttowhichLLCmembersare
effectivelymeetinglearningoutcomes(AppendixA)andacquiringspecificcompetencies
relatedtosocialchangeandleadership.Thisformativeassessmentwillprovidequalitativeand
quantitativedatathatwillshedlightonthe"program'soverallworthormeritinrelationto
importantcriteria"(Fitzpatricketal,2011,p.21).Inthisstudy,the
importantcriteria
Fitzpatrickisreferringtowouldbethelearningoutcomesofthelearningcommunity.
Asstatedbefore,previousevaluationsadministeredbyLoyolaResidenceLifehaveprimarily
askedquestionsregardingcommunityengagementandoverallexperienceparticipatingina
learningcommunity.Whilethesequestionsarebeneficialtoevaluatingtherelationalcomponent
ofalearningcommunityexperience,theydonotexplicitlyinformstakeholdersoftheacquisition
ofcompetenciesstatedinLLCspecificlearningoutcomes.Focusgroupshavealsobeenutilized
toevaluatetheexperienceofLLCmembershowever,similartothequantitativesurveys,the
resultsreflectthegeneralsatisfactionofparticipants.Bycreatedanewevaluation,evaluatorsare

MEETINGTHELLCOUTCOMES

abletoimplementinstrumentsthatwillproducerelevantdatatobeusedbyvariousstakeholders
withvaryinginterestintheLLC.
Stakeholders
AsBrysonandPatton(2010)pointout,itisimportantforevaluatorstooptimizethe
intendeduseofevaluationsfortheintendedusers(Wholeyetal.,p.30).Astakeholderanalysis
wasconductedbyevaluatorstoillustratethelevelofpowerorinterestofeachstakeholdersin
relationtotheLLC.Stakeholdersinthisprogramrangefromfirstyearstudentstoseniorlevel
professionalsatLoyolaUniversityChicago.Usingthepowerversusinterestgrid,atwobytwo
matrixthatdisplaysastakeholderspoliticalinterestandpowertoaffecttheevaluationprocess
anddataapplication,evaluatorsareabletogridthelevelofinterestandpowerofallstakeholders
involvedintheevaluation(Bryson&Patton,2010).Thesestakeholdersinclude:JaneNeufield
(DivisionofStudentDevelopment),MarciWalton(ResidenceLife),ShannonHowes(Student
LeadershipDevelopment),LLCmembers,theresidencehalldirector,theLCAs,LCCfaculty,
andstudentswithintheresidencehall.
TheVicePresidentforDivisionofStudentDevelopment(DSD),JaneNeufieldqualifies
asastakeholderofhighpower(highpowerininfluencedataevaluation)andlowinterest(low
politicalinterest).Jane,alongwithhercolleagues,createandsetthetoneforthemissionand
goalsofthedivision,especiallyintheareaofstudentengagement.ThoughJanedoesnotwork
functionwithinResidenceLife,theDirectorofResidenceLifereportsdirectlytotheVice
PresidentofDSD.Ontheoppositeendofthespectrum,stakeholderswithlowpowerandhigh
interestconsistofthestudentswithintheleadershiplearningcommunities.Otherstakeholdersin

MEETINGTHELLCOUTCOMES

thematrixcategoryoflowpower,highinterestincludetheresidencehalldirector,LCAs,andLC
faculty.ThefacultyinstructthethreecoursesLLCmembersarerequiredtoenrollintogether
duringtheacademicschoolyear.Theseindividualsprimarilyworkwiththelogisticsofthe
programsuchasoverseeingtheLLCbudget,programming,cofacilitating,andadvocatingfor
potentialconcerns.
ShannonHowesandMarciWaltonaretheonlytwostakeholdersthatshareboth,high
interestandhowpower.AstheAssistantDirectorforAcademicSupportandLearning
Communities,Marcioverseestheoperations,learningoutcomes,andlogisticsofallseven
learningcommunities:Greenhouse,International,Multicultural,Service&Faith,Wellness,
FirstYearResearchExperience,andLeadership.Shealsoischargedwiththeresponsibilityof
promoting,coordinating,andmemberplacementforthelearningcommunities.ShannonHowes,
astheDirectorofSLDandAdministrativePartneroftheLLC,partnerswithMarcitocreate
cocurricularactivitiesthatarebasedonthelearningoutcomesformembersoftheLLC.Inher
roleasanAdministrativePartnerShannonalsocoadvisedLCAsandcollaborateswithresidence
lifetocreateavisionandstrategyforstudentparticipation.AsFitzpatrick(2010)pointsout,"
theprimarypurposeofevaluationistorenderjudgmentsaboutthevalueofwhateverisbeing
evaluated"(p.13).ForShannonandMarci,resultsfromthisevaluationwillprovetobea
constructivetoolinassessingthevalueandimpactoftheLLC.
Finallystakeholdersthatareconsideredthecrowd,havelowinterestandlowpower.In
thiscategoryaretheresidentsandstudentstaffthatarenotrequiredtoparticipantintheLLCbut

MEETINGTHELLCOUTCOMES

10

sharespacewithintheSimpsonresidencehall.Belowallstakeholdersareoutlinedinthepower
versusinterestgrid:
Players(Highpower,Highinterest)
MarciWalton,AssistantDirectorof
AcademicSupport&Learning
Communities
ShannonHowes,DirectorofStudent
LeadershipDevelopment&
AdministrativePartner

Subjects(Lowpower,Highinterest)
LeadershipLearningCommunity
Members
LearningCommunityAssistants
ResidenceHallDirector
LCFaculty

ContextSetters(Highpower,Lowinterest) TheCrowd(Lowpower,Lowinterest)
JaneNeufeld,VicePresidentforthe
Residentswithinthehall
DivisionofStudentDevelopment
StudentStaff:ResidentAssistants
(RAs)

LogicModel

Itisimportanttousealogicmodeltoproperlyevaluatetheprograminordertoseethe

complexitiesoftheLLCasawhole.Alogicmodelisaplausibleandsensiblemodelofhowa
programwillworkundercertainenvironmentalconditionstosolveidentifiableproblems
(Wholeyetal,p.56).Thelogicmodel(AppendixB)thathasbeendevelopedlooksatthe
LeadershipforSocialChangeLearningCommunity(LLC)pertainstothetwomaindepartments
overseeingitsoperationsResidenceLifeandStudentLeadershipDevelopment.Thismodel
assessesthepartnershipsandhowbothinfluencetheoutcomesthatwillbeevaluatedthroughout
thisproject.

Themodelbeginsbylookingattheinputsofeachdepartmentsuchashumanor

financialresourcesandactivitiesthataretheessentialactionstepsnecessaryfortheprogramto

MEETINGTHELLCOUTCOMES

11

takeplace.Thesearebrokendownintothreecategories:staffing,time,andfinancial.The
staffingincludesbothstakeholdersMarciWaltonandShannonHowes,aswellasotherpeople
whodirectlyinteractwiththestudentsinthelearningcommunity.Eachdepartmentprovides
differentresourcesthatenableoutputssuchasprogrammingrelatedtopersonalleadershipstyles
andgettingtoknowChicagoforthebettermentofthestudentsinvolved.Inaddition,themodel
alsodisplaystheshort,medium,andlongtermgoalsoftheprogram.Shorttermgoalsarein
placetokeeptheprogramfunctioningandworkingtowardsgreatergoalstheseincludethe
programsthateachdepartmentoffersandbroadlearningcommunityoutcomeslikefosteringa
communitybetweenthestudentsintheLCandthestaff.Overtime,involvementissupposedto
developstudentsintomoreacademicallyandsociallystrongerindividuals,withagreatersense
ofwhattheirdefinitionofleadershipis.

Supportingthemodelareassumptionsabouttheprogramandexternalfactorsthataffect

studentsuccess.Itisimportanttoqualifythesebecausetheymayimpacttheevaluation.The
assumptionsthatwereidentifiedwerethatallstudentswereparticipatingintheeventshostedby
SLDandResidenceLifeaswellasthatthecommunityispositiveandencouragesgrowth.If
studentsarenotparticipatingintheevents,theyarenotreceivingvaluabletimetoreflectontheir
definitionofleadershipandengagingwiththegroupsimilarly,ifthereisanegative
environment,studentsmaybelesslikelytopushpasttheircomfortzone.Theexternalfactorsare
whetherstudentsapplyfortheprogramatall,thechallengesofmaintainingacademic
coursework,andtheacademicadvisingthataccompaniestheprogram.Forstudentswhoare
strugglingintheircoursework,theymaybeaskedtoleave,resultinginashiftinhowthe

MEETINGTHELLCOUTCOMES

12

communityinteractswitheachotheroriftheyareperceivedliketheyarenotcontributing.All
oftheseimpactpotentialresultsoftheevaluation.
QuantitativeApproach
InordertoexaminetheextenttowhichtheLeadershipLearningCommunityishelping
studentmemberseffectivelymeetlearningoutcomesandacquirespecificcompetenciesrelated
tosocialchangeandleadership,evaluatorsmustfirstcollectquantitativedatafromfirstyear
LLCmembers.Thepurposeofthequantitativeevaluationapproachistoestablishandassess
criteriathatmeasuresstudentcompetencyinthreeareasrelatedtoleadershipdevelopment:
practicalapplicationculturalcompetencyandcurriculumandcocurriculum.Theadvantageof
assessthesethreeareasinaquantitativeformatistheabilitytomeasurestudentsperceptionsof
theirowngrowth.Forhighpowerstakeholders,quantitativedatacanplayalargerolein
determiningfundingandbudgetallocations.Inthisportionoftheevaluation,thefollowing
sectionswillbeaddressedthequantitativeresearchdesign,methods,dataanalysisand
quantitativedatapresentation.
Population&SamplingFrame
Participationinthestudywillbesolicitedfromapproximately100firstyear
undergraduatestudentsintheLLC.Evaluatorsplantoutilizeeachmemberinthispopulationfor
itssample,creatingthemostaccuratedata.Studentswillbeinvitedtocompleteapenciland
paperinpersonsurveysduringmandatorypredeterminedmeetingsandprogramming.
AccordingtoNewcomerandTriplett(2010),pencilandpapersurveyshavehigherresponse
ratesifparticipantsarealreadyrequiredtobeinalocationorspace(Wholeyetal,2010).

MEETINGTHELLCOUTCOMES

13

ThoughthesurveyswillbedistributedateventsthatalreadyrequiresattendanceforLLC
students,theywillnotbemademandatorybyevaluators.Thegoalistoincreasetherateof
participationandcompletionofthesurveybyofferingthesurveyinpersonateventswith
maximumLLCmemberattendance.Atthebeginningoftheacademicyear,thepresurveywill
begivenduringtheirAugustLeadershipLearningCommunityorientation,aswellasduringa
finalcommunitymeetingfacilitatedbyResidenceLifeattheendoftheyearinMayforthe
postsurvey.ThereturnrateispositivelyimpactedbystudentsinitiallyoptingintotheLLCand
iscoupledwiththepresenceofthesurveyattwohighattendanceevents.TheLearning
CommunityAssistantwilltouchbasewithstudentsthatmisseitheroftheseeventsandwilloffer
LLCmemberstheopportunitytopartakeinthestudyiftheydesireto.Surveyswillbe
distributedbythreetofourLLCfirstyearstudentvolunteerstoensurethatallpresentLLC
receiveasurveyinatimelymanner.Staffwillexittheroomwhilethesurveyisbeingcompleted
andturnedintoaboxlocatedneartheexitattheendtoenhanceprivacyandencouragemore
authenticresponses.However,onceexitingtheroomevaluatorsandstaffwillnotbeableto
ensurethatallstudentsreceivedorcompletedasurveyuponleavingthemeeting.Toeliminate
potentialsurveyfatigueafterameeting,evaluatorspaidcloseattentiontotheresearchdesignand
method.
ResearchDesign&Method
ToevaluatetheeffectivenessoftheLeadershipforSocialChangeLearningCommunity
(LLC)ininsuringthatfirstyearmembersaremeetinglearningoutcomessetbytheprogram,a
quasiexperimentalpretestposttestdesignwillbeadministered.Aquasiexperimentaldesignis
mostappropriateforthisstudyduetotheexistenceofconfoundingvariableswhichmakethe

MEETINGTHELLCOUTCOMES

14

LLCpopulationnonequivalenttootherlearningcommunitiesorstudentcommunitieson
Loyolascampus(Johnson&Christensen,2012).Additionallythisdesignallowsevaluatorsto
focusonasinglegroupandtheeffectsofagiventreatmentorinterventionduringaspecifictime
andspace.Inthisstudythesinglegroupwillconsistofallfirstyearstudentswhoparticipatein
theLLC.Thelearningcommunitywilloperateastheinterventioninthisdesign.Figure1.1
illustratesthedesignutilizedinthestudy.Inthefigurebelow:O1representsthedistributionof
thepretestsurveytofirstyearLLCmembersbeforetheintervention,Xrepresentsthe
intervention(theLLCprogram),andO2representsthedistributionoftheposttestsurveyafter
theintervention.
Figure1.1
LeadershipforSocialChangeLearning
CommunityFirstYearStudents

O1XO2

Sincethestudentsareallindividualswhohaverequestedtobeinthisspecificlearning
communityduringthe20142016academicschoolyearandarenotrandomlyassignedtothe
samplingsizeofparticipants,thisstudyisnotatrueexperiment(Creswell,2009).Inadditionto
notbeingselectedatrandom,LLCstudentswillnotberandomlyassignedtotakethesurvey.
Basedontheuniqueandspecificcharacteristicsofthegroup(firstyearLoyolastudents
participatingintheLLC),usingacontrolgroupisimpracticaltothepurposeoftheevaluation
sincenocomparablegroupofpopulationexistsonLoyolascampusduetothecustomized
curricularandcocurricularexperienceintheLLC.

MEETINGTHELLCOUTCOMES

15

Togaincomprehensiveresultsfromthesurvey,acensussamplingwillbeemployed.A
censusisasamplingoptioninwhichanentirepopulationisaskedtoparticipateinasurvey.This
typeofsampleismostusefulwhenevaluatingasmallpopulation(Newcomer&Tripplet,2010).
Inthisevaluation,responseswillbesolicitedfromallfirstyearLoyolastudentswhoare
currentlyenrolledintheLeadershipforSocialChangeLearningCommunity.Forthisstudy,the
samplingsizeisapproximately100studentssinceevaluatorsplantoutilizetheentireLLC
firstyearstudentpopulation.Inanticipationforsamplingsizeofapproximately100students,
thesurveyinstrumentwasdesignedtoallowevaluatorstocodedatacollectinanefficient
manner.
SurveyInstrument
Thisevaluationwilluseanoriginalsurveyforthepreandposttest.Theinstrumentwillbea
Likertstyleandopenanswersurveycontaining13questionsrelatedtostudentsselfreported
competencyintheareasofsocialchangeandleadership.Acontinuousscalewillbeutilizedto
measurestudentsresponsesontwelveoutofthirteenquestions,participantswillbeabletoreport
theirlevelofconfidenceandagreementwithitemslistedonthesurvey(Creswell,2009).As
recommendedbyDillman(2007),thesurveycontainsanequalnumberofpositiveandnegative
scalarresponses,decreasingpotentialbiastoskewtheresultsinnegativeorpositivedirection.
Thesequestionsaredividedintothreedifferentcategories:PracticalApplication(applyingthe
competenciesgainedinpersonalandprofessionalexperiences)CulturalCompetency(awareness
andunderstandingofvaryingculturalbehaviors,traditions,andperspectives)andCurriculum&
Cocurriculum(requiredacademiccoursesforthelearningcommunityandprogramsfacilitated

MEETINGTHELLCOUTCOMES

16

byResidenceLifeandStudentLeadershipDevelopment).Finally,thesurveyconcludeswitha
demographicsectionthatrecordstheparticipant'sraceandgender.
Thepurposeofthepretestistoestablishthestudent'slevelofcompetenceintheareasof
socialchangeandleadershipthatstudentsbringwiththempriortoparticipatingintheLLC.
Simultaneously,thepretestwillallowevaluatorsandstakeholderstounderstandincoming
studentsawarenessorfamiliaritywiththesetwocompetencies.Thesurveywilltake
approximately58minutes,thelatterbasedupontheresponsetothequalitativequestionatthe
endofthesurvey.Theinformationcollectedonthepretestwillbecomparedtotheposttest
resultsattheendoftheacademicschoolyear.
Theposttestwillfollowthesameformatasthepretestwithminimumadditions.For
example,inthepretest(seeAppendixC)studentswillread,Iamconfidentinmyabilityto,
whereastheposttest(seeAppendixD)willread,BasedonparticipationintheLeadership
LearningCommunity,Iamconfidentinmyabilityto.Thepurposeoftheposttestistomeasure
thelevelofgrowthinthethreesurveycategoriesastheyrelatetotheleadershiplearning
outcomes.
Pretestandposttestwillbedeliveredinpersonatthefirstandlastmeetingsoftheschool
st
th
year.Thesemeetingswilltakeplaceonthe1
and29
weekofclassandareusuallymandatory

forallmembers,thusincreasingthepotentialforahighcompletionrateamongsurvey
participants.Theutilizationofthepencilandpapersurveyswillyieldhigherresponseratesof
participantsalreadyattendingthemeeting(NewcomerandTriplett,2010).Bothpretestand
posttestwillbeanalyzedbyanevaluatoroncecompleted.

MEETINGTHELLCOUTCOMES

17

Statistical&DataAnalysis
Ananalysisofvariance(ANOVA)andttestwillbeemployedafterdatacollection.
CooperandShelleyexplainthatANOVAismostappropriatewhencomparingmeanlevelsofa
continuousdependentvariableacrossthelevelsofacategoricalvariable(Schuh,2009).Since
thisstudyincludesvariousdependentvariableswithinthreecategoriesofthesurvey,(practical
application,culturalcompetency,andcocurriculum&curriculum)amultivariateanalysisof
variancewouldallowfutureevaluatorsdatatofurtherinvestigateformativerelationshipswithin
theLLC.
Forcoding,thedatavaluesforallscalarquestionswillrangefromonethroughfive.For
example,notconfident=1andveryconfident=5.Thiswillprovideaconsistentinterpretationof
thedatabyallevaluators.Implementingthesevariablesintoapairedsamplesttestforthe
pretestandposttestwilldemonstratetheextentatwhichtheprogramiseffectiveinthe
developmentofcompetencieswithinLLC.Whileusingthepretestandposttestmethodis
appropriateforthegoalsofthisstudy,itisacknowledgedthattheexperienceoftakingapretest
mayalterpotentialresultsontheposttest(Johnson&Christensen,2012).Demographicswithin
thesurveywillalsobrokendownforcodingpurposes.Raceandgendercategorieswillbe
associatedwithnumericalvaluessimilartothescalarquestions(Male=1,Female=2,andNot
Listed=3).Eachsurveywillbeenteredintoanonlineformbytheevaluatorsinordertohelp
organize,analyze,andchartthedata.Forstudentswhodonotidentifywithabinarygender(i.e.
maleorfemale),evaluatorswillsupplyaNotListedoptionwherestudentscanfillinhowthey
identity.Thefinalopenendedquestionwillaskstudentstolistleadershipopportunitiesthatthey
areawareofatthetimeoftheirparticipationinthesurvey.Thisquestionisdesignedtobeinline

MEETINGTHELLCOUTCOMES

18

withoneofthelearningoutcomescreatedbytheLLCwhichasksstudentstodescribeatleast4
ofthedifferentprograms,services,andopportunitiesoncampus(andbeyond)wheretheycan
connectandengageintheleadershipprocess.
ValidityandReliability
Toincreasereliabilityofcodingandanalysis,evaluatorswillbeagreeinguponthelevelof
consistencyforinterpretationofthequantitativedata.Inotherwords,evaluatorswillbeutilizing
interscorerreliabilitybycodingthedataindependentlyandthencomputethecorrelatebetween
theinternalandexternalevaluator(Johnson&Christensen,2012).Also,evaluatorswillincrease
reliabilitybyavoidinginteractionsbetweenresidencelifestaffandLCCstudentswhilesurveys
arebeingcompletelyandreducetheperceptionofresidencelifecoercion.Toprovidevalidity
evidenceempiricalandtheoreticalrationalesthatsupporttheinferencesorinterpretationsmade
fromtestscores,evaluatorswillbeusingcontentrelatedevidence(p.144).Thequestionsand
contentofourevaluationwillbeconfirmedbystakeholderstoensurethattestquestions
adequatelyaddresstheinterestofthosewithhighpowerorhighinterest(refertostakeholders
section.)
QuantitativeDataPresentation
Theresultsofthequantitativesurveywillbecompiledintoabriefandsharedwith
stakeholders,ShannonHowes(DirectorofStudentLeadershipDevelopment)andMarciWilton
(AssistantDirectorforAcademicSupportandLearningCommunities),duringascheduled
appointmentoncetheacademicyearhasendedduringMayorJune.Resultsofthesurveywillbe
turnedintolinegraphswiththepreandposttestresultssuperimposedtochartgrowthbetween

MEETINGTHELLCOUTCOMES

19

thetwotests.Thebriefwillsummarizethegeneralfindings,explainthelinegraphswherethere
arediscrepanciesinquestionphrasing,andincludeallofthecompletedsurveysattachedtothe
backsothattheybothcanrefertooriginaldocuments.Anyrecommendationsfromstudents
collectedduringthequalitativedatacollectionwillbeincluded,asanadditiontothequantitative
data,usingbulletpointslistedintheorderofhighestrateofoccurrencetotheleast.Stakeholders
willbeabletousethedatatoinformpracticesmovingforwardandcancontacttheevaluatorsif
theyhaveanyquestions,comments,orconcerns.
QualitativeApproach
AsSchuh(2009)pointsout,regardingtheuseofquantitativeandqualitativestudies,
Whereasbothquantitativeandqualitativeapproacheshavetheirownuniqueadvantagesand
challenges,theycanbeusedtocomplementeachotherandtoprovideamoreholisticapproach
(p.127).Byincludingaqualitativecomponent,stakeholderswillreceiveabreadthofknowledge
specifictowhatoutcomesweremostsalientforindividualsthatthequantitativeportioncould
not,aswellasdelvedeeperintotheirsurveyresponses.
Whiletherearemanyqualitativemethodsinwhichtoselectfrom,therearemany
strengthsinoptingtousefocusgroupsbecauseoftheirabilitytocombineparticipant
experiencesanduncoverfactorsthatinfluenceopinions,behavior,ormotivation(Kreuger&
Casey,2000).Othermethodsmightprovidemoreindividualizedassessment(interviews)orbe
designedtoexplainstudentsexperiencesingreatdetail(openendedquestionnaires)however,
focusgroupsengageeighttotenstudentsinaconversationaroundatopic,whichmodelsthe
communitylearningthathastakenplaceintheresidencehallsandclassroomsthroughoutthe

MEETINGTHELLCOUTCOMES

20

year.Theevaluatorshopethatthesefactorswillhelpfosterasafespacewherestudentsfeelmore
comfortabletosharepersonalexperiences.
FocusGroupParticipants

AllstudentsintheLeadershipLearningCommunity(LLC)willbeeligibletotakepartin

thefocusgroupstudybecausetheywillhavehadafullyeartoprocessandhaveexperiences.
Conveniencesamplingwillbeimplementedtogatherparticipants.Schuh(2009)mentionitis
easiertorecruitandincreasethelikelihoodofattendancethroughthisapproach.Othertypesof
sampling,includingclusterandprobabilitysampling,werenotselectedbecauseofthesmaller
samplesizewithminimalstratificationordifferentiating.Conveniencesamplingmayraisethe
riskofbiasfromtheparticipantsandunpredictableresponseshowever,sincethestudyis
lookingspecificallyattheLLCparticipantexperience,itisonlytryingtoreachaparticular
individual.Thestudentsareallgatheredinclassesthatareonlyavailabletothem.Inperson
signupswillbemadeconductedbytheinternalevaluatorattheendoftheLLCstudentsclass
hourstwoweekspriortothefocusgroups.
Thegoalistorecruitsixtoeightparticipantsforfourfocusgroupsifinterestishigh,the
firsttennamesthatsignupwillbetakenforeachfocusgroup,andtheextranameswillbesaved
incaseoflowturnoutintheotherfourfocusgroups.KruegerandCasey(2010)recommendthis
focusgroupssizeforsensitiveorpersonalorwhentheparticipantshaveconsiderableexpertise
orexperiencewithatopic(p.382).Threefocusgroupswilltakeplacefifteenminutesaftereach
classletsouttogivetimeforparticipantstousethebathroomandhaveasmallbreak.Incentives,
intheformofDominospizzaanddrinks,willbeprovidedtoalleviatetheneedsofstudentswho
mightnothaveparticipatedotherwise,assuggestedbyKruegerandCasey(2010).Afourth

MEETINGTHELLCOUTCOMES

21

focusgroupwillbeofferedintheeveningforthosewhowerenotabletotakethetimeaftertheir
classduetoothertimeconstraintsandresponsibilities.
FocusGroupProtocol

Thefocusgroupswillattendtoavarietyofexperiencesthatcoverthelearningoutcomes

thatwerepresentedbyStudentLeadershipDevelopment(SLD)andResidenceLife.Drawing
fromKruegerandCaseys(2010)recommendations,thetenfocusgroupquestionsaredraftedas
openendedinordertoelicitdescriptionsofhowthestudentsleadershiplessonshavebeenused
(ornot)indifferentcontextssincethestartoftheyear:howtheimplementedcurriculumhas
impactedtheirleadershipdevelopmentandwhetherornotculturalcompetencieswere
effectivelyaddressedbyfacultyinstructorsandtheLearningCommunityAssistant(LCA).
Studentswillbeallowedtoenterthespace,grabpizzaoradrinkpriortothestartofquestioning,
andengageininformaldialoguewiththemoderatorspriortothestartofthefocusgroup.This
willallowparticipantstobecomemorecomfortableinthespaceanddeveloparelationshipwith
thenewindividualstheywillbesharingwith.
Thefocusgroupwillstartwithashortintroductionofthepurposeofthestudy,writtenin
ourfocusgroupprotocol(AppendixE),andwillbefollowedbyanopportunitytosigntheFocus
GroupConsentForm(AppendixF).Eachsectionofthefocusgroupwillrelatetooneofthe
learningoutcomesbutwillnotbeexplicitlystatedassuchthiswillhopefullylimittheamount
ofanswerstiedtoperformativity.Performativity,inthissituation,iswhenstudentswouldtryto
provideanswerstheybelievetheevaluatorswanttohearortryingtoliveuptoapreconceived
standard.Thefocusgroupwillconcludewiththemoderatorofferingthechanceforanywelcome
feedbackthattheywerenotabletoshareduringthefocusgroupifquestionsareexhaustedordo

MEETINGTHELLCOUTCOMES

22

notcomeup,themoderatorwillthankeveryoneandgivetheremainderofthepizzaanddrinks
toanyonewhowantsit.Weareanticipatingthateachfocusgroupwilllastaboutanhourtoan
hourandfifteenminutes.
FocusGroupImplementation

Theinternalevaluator(DeNeatriaRobinson)willbetaskedwiththerecruitmentand

schedulingoffocusgroupparticipantsduetohercurrentroleasanAssistantHallDirectorat
LoyolaUniversityChicago.Theabilitytooperateoncampusandtohavesomefamiliaritywith
thestudentsintheLLCwillprovideherthebestchancetohaveindividualssignup.Theinternal
evaluatorwillcontactprofessorstoaskwhethershewouldbeabletogointoclassroomsand
elicitstudentparticipationinthefocusgroupstogaininitialinterest.Emailswillbesentoutto
studentsusingtheemaillistprovidedbythestakeholders,theAssistantDirectorandtheDirector
ofStudentLeadershipDevelopment,inthecaseprofessorsareunwillingtoallowclasstimeto
becoopted.Studentsexpectvaluableinformationsinceregularemailsaresenttothoseinthe
LLC.Duringtheschedulingperiod,nameswillbegatheredandassignedtoafocusgroup.The
focusgroupwilltakeplaceafterstudentstakethequantitativesurveysbutbeforefinalsto
removesomeoftheriskoflowsignupsandwillbeinoneoftheclassroomsontheLakeshore
Campus.
Theroomwillbesetupinacircletoalloweveryonetoseeeachotherandfeellikeapart
ofthegroup.Oncestudentsarrive,theinternalevaluatorwilltransitiontoassistantmoderatorto
takeasdetailedofnotesduringtheconversation.Theexternalevaluator(EricRuelle)willbethe
primarymoderatorintheroomduringthefocusgroupsandleadthedatacodingprocess.The
externalevaluatorwillbeleadingthediscussionbecauseheisnotassociatedwithLoyola

MEETINGTHELLCOUTCOMES

23

UniversityChicagosResidenceLife,whichwillpromptmoreclarifyingquestionsfor
understandingandmakethestudentsfeelmoreliketheexpertsintheroom.Focusgroupswillbe
audiorecordedbytheinternalevaluatorhowever,bothmoderatorswilltakenoteswiththe
internalevaluatorassignedtononverbalcuesandtheexternalevaluatorconcentratingonthe
contentofwhatparticipantssay.
AnalysisoftheData
Thedatawillbeanalyzedusingasystematicanalysisprocess,whichWholey,Hatry,and
Newcomer(2010)definesaswhentheanalysthasaprotocolthatfollowsapredeterminedand
verifiablesetofsteps(p.396).Partialanalysiswillstartwhilethefocusgroupisstillhappening
witheachmoderatornotingdifferentverbalandnonverbalcuesofparticipants.Afterthefocus
groups,moderatorswillgooverthenotestheytookandwritebriefsummariesofthecontent.
Theevaluatorswillcontinuetoaudiorecordanyconversationregardingtheirobservationsto
createarunninglogandmemorybankforanythoughtsthatmightcomeupandmaynotbe
writtendown.Therecordingofthefocusgroupsandnoteswillbetranscribedatalaterdate.
Transcriptionforthefocusgroupwillbeseparatedintolinesthatsaidbythemoderatorandwhat
issaidbytheparticipants.Codingwillbeginafterthatandresultinidentifyingthemesand
trendsregardingthelearningoutcomesoftheLLC.Timewillbeafactorincoding,butthe
summervacationwillofferthechancetoanalyzethedataandcomebacktothestakeholders
withinformationthattheycanthenusetopotentiallymakeimprovements.Anotherexternal
evaluatorwillalsobeutilizedduringthecodingprocessinordertoincreasevalidityforthiswe
willaskClaireBailey,GraduateAssistantinLoyolasStudentActivitiesandGreekAffairs,who
istangentiallyfamiliarwithLLCandhasverylittlestakeintheprogram.Thisincreasesvalidity

MEETINGTHELLCOUTCOMES

24

becauseitallowsdifferencesinopinion,experience,andbiastobecomemoreconsistentor
standouttoberemoved.Thestakeholderscanalsobereassuredthattheinformationwillstay
internalbecauseClaireworksatLoyolaandhaslittlestakeatotheruniversitiesordepartments.
CodingProcess
Codingis,theprocessoforganizingthematerialintochunksorsegmentsoftextbefore
bringingmeaningtoinformation(Creswell,2009,p.186).Sincemanyofthequestionsare
structuredaroundthelearningoutcomes,thecodingmethodwillbeginwithlookingforthemes
inagenericsystemicprocessasdescribedbySchuh(2009)toseparateoutcomesfromeach
other.Directquoteswillonlybeusedtofurtherelaborateoncommonthemes.Whileitis
importanttoseethelearningoutcomesrepresentedinthecollectedresponses,evaluatorswill
alsoneedtoassignapositive,negative,neutral,orunidentifiablemarkwhentrackingthe
learningoutcomesbecausewearetryingtoseehowtheLLChasimpactedthesequalitiesfor
participants.Stakeholderswillhelpdevelopthecodesforfocusgroupresponses.Eachfocus
groupwillhavetheirfindingssavedinanexcelfilewiththeselectedphrasesappearingina
tchartwiththeheadings:positive,negative,neutral,orunidentifiable.Thecodingwillbe
conductedbythetwoevaluatorseachtakingtwofocusgroupstotranscribeandcodeandthen
switchingtodoublechecktheworkoftheother.Additionalthemesthatmaybeidentifiedfrom
theparticipantsresponseswillbecodedandshadedadifferentcolortoprovidethestakeholders
withusefulinformationonwhattheyfoundmostuseful.
Validity&Reliability

Wewillimplementmeasurestoensurethatpropercodinghastakenplace.Havingan

internalandexternalevaluatorconductthefocusgroupandcodingwilldecreasethelikelihood

MEETINGTHELLCOUTCOMES

25

ofpersonalbiasandlimitthewaythatoneevaluatormighthavecontrolovertheotherones
findings.CodeswillbedevelopedwiththeinputofstakeholdersMarciWhalton,Assistant
DirectorofResidenceLifeandLivingLearningCommunities,andShannonHowes,Directorof
StudentLeadershipDevelopment.Asapointofconsiderationduringthefocusgroups,itwillbe
importantforthemoderatorconductingthediscussiontoinitiateamembercheckinordertostart
thecodingprocessforpositive,negative,orneutraltraits.Amembercheckis,takinginitial
interpretations,findings,anddescriptionsbacktotheparticipantsfromwhomtheywerederived
todeterminewhethertheyareaccuratefromtheparticipantspointofview(Schuh,2009,p.
169).Additionally,peerdebriefingwillbeutilizedsosomeonewhoisnotfamiliarwiththestudy
(ClaireBailey)canreviewthefindingsandaskquestionsregardingthestudy.
Limitations
Thoughhavingtwoevaluatorswithexperienceinresidencelifemaysupplyafoundation
foradequateunderstandingofthebasicoperationswithinalearningcommunity,evaluator
familiaritymaystillcreatebiaswithinquestionconstructionandfocusgroupmoderation.
Informationcodedbythefocusgroupmaybeinterpreteddifferentlydependingonthe
evaluatorsexperiencewithresidencelife.ThisisespeciallytrueforDeNeatriaRobinsonasan
internalevaluatorwhoworksforthedepartmentinwhichthefocusgroupisbeingcarriedout.It
willbeneutralizedbyhavingtheexternalevaluator,EricRuelle,moderatethefocusgroups.
Groupdynamicsisanotherlimitationwithinthefocusgroup.Thegrouphaspotentialto
reflectthedemographicsofLoyolaUniversityChicagofirstyearpopulation:majorityWhite
(1,227outof2,194)andmale(1,468outof2,194)(LoyolaUniversityChicago,2015).Thus
creatinganenvironmentthatmayimplicitlysilencetheexperiencesofotherstudentswhodonot

MEETINGTHELLCOUTCOMES

26

sharetheseidentities.Evaluatorsmustremaincautiouswhenwritingandcodingdataabout
studentpopulationsthathavebeentraditionallyunderrepresented(Schuh,2009).Additionally,
evaluatorsmustacknowledgethatthefocusgroupparticipantsallliveinonecommunity,
thereforestudentsmaybemoreorlessopendependingonthecultureofthelearningcommunity.
Finally,thoughtheassessmentseekstogatherinformationabouttheacquisitionof
specificcompetencies,stakeholdersinresidencelifehavelimitedcurricularinvolvement.For
example,studentscansharethatthecurriculumoftheLLCsrequiredcoursesbarelycontribute
totheirunderstandingofsocialchange.ResidenceLifecanrelaythefeedbacktofaculty
however,StudentLeadershipDevelopmentorResidenceLifecannotmakeindependent
alterationstothecurriculumandsyllabus.Thus,thepurposeofourdataisnottomake
generalizationsaboutalllearningcommunities,buttounderstandthecontextualcomponentsthat
contributetothestudentexperiencewithinLoyolasLeadershipLearningCommunity.
DataPresentation
Sincethedataiscollectedthroughaqualitativeapproach,themeswillbepresentedina
descriptivenarrativeform.Thefocusgroupresultswillbepresentedintheformofanexecutive
reportpreparedbyEricRuelleandDeNeatriaRobinson.Thecodeddatawillbeformattedinto
areportwhichwillgroupprevalentthemesandsharedexperiencesfromthefocusgroups.To
addressoneofthelimitationsstatedearlier,wewilltakeanaccountofdramaticallycontrasting
experiencesandfeedback,beingsuretonotusedirectquoteswhichmaysuggesttheidentityof
specificparticipants.Directquoteswillonlybeusedtofurtherelaborateoncommonthemes.
Tabularforms,asoutlinedbyCreswell(2009)willbeusedtocreateagraphicillustrationofdata

MEETINGTHELLCOUTCOMES

27

collectedinthefocusgroup.Evaluatorswillcreatetablecomparingsimilarandcontrastingcodes
orthemes.
Budget
ThemanyneedsoftheevaluationprojectrequirethemonetaryassistanceofStudent
LeadershipDevelopmentandResidenceLifetoprovidethestakeholderswithasdetailedaplan
aspossible,theevaluatorssubmittheattachedbudget(AppendixH).Thebudgetprovidesas
closetoafinalaccountingofwhatisneededthroughoutthecourseoftheyear.Costshavebeen
minimizedasmuchaspossiblehowever,somecostsneededtoremaininordertohave
incentivesfortheparticipantsofthefocusgroupandadministrativeresourcesliketheprintingof
surveys.PersonnelcostsareincludedintheassistantshipstipendsthatareprovidedbyLoyola
UniversityChicago.Theexternalevaluator(EricRuelle)istheonlypersonnelnotcoveredbythe
providedstipendbuthaswaivedhisfeefortheexperience.
NextSteps
ThisevaluationseekstoexaminetheextenttowhichLLCmembersareeffectively
meetinglearningoutcomeandacquiringspecificcompetenciesrelatedtosocialchangeand
leadership.Thiswillbedonebyimplementingthequantitativeandqualitativeapproachlaidout
intheevaluationplan.Afterthedataiscollectedandpresentedtostakeholders,MarciWalton
andShannonHowes,evaluatorsplantoutilizethedataforthreeadditionalpurposesdisplayedin
theposterpresentation(AppendixI).First,tracktheeffectivenessofthecocurricularand
curricularcomponentsoftheLeadershipLearningCommunity.Second,makerecommendations
tothestakeholdersbasedonstakeholderinterestanddatafindings.Finally,evaluatorswillbe
abletoassesstheLCCexperienceforfirstyearstudents.

MEETINGTHELLCOUTCOMES

28

Additionally,evaluatorshopetoencouragemorespecificassessmentsoflearning
communities(LC)inLoyolaResidenceLife.ResidenceLifeandvariousstakeholdersinother
learningcommunitiescanbenefitfromcreatingassessmentsthattargetlearningoutcomesin
additiontotheoverallLCexperience.Stakeholderscanalsousethedatapresentedtoadjust,add,
oreliminatedifferentlearningoutcomes.

MEETINGTHELLCOUTCOMES

29

References
Creswell,J.W.(2009).
Researchdesign:Qualitative,quantitative,andmixedmethod

approaches(3rd

Edition)
.ThousandOaks,CA:SagePublications.

nd
Dillman.D.A.(2007).
Mailandinternetsurveys:Thetailoreddesignmethod
(2
ed.).

Hoboken,NJ:Wiley
Johnson,R.B.&Christensen,L.(2012).
Educationalresearch:Quantitative,qualitative,and
th
mixedapproaches
(4
ed.).Boston,MA:SagePublications.

Krueger,R.A.&Casey,M.A.(2010).Focusgroupinterviewing.InJ.S.Wholey,H.P.Hatry,
rd
K.E.Newcomer(Eds.),
Handbookofpracticalprogramevaluation
(3
ed.).

SanFrancisco,CA:JosseyBass.
Schuh,J.H.&Associates(2009).
Assessmentmethodsforstudentaffairs
.SanFrancisco,CA:
JosseyBass.
LoyolaUniversityChicago.(2015).Officeofinstitutionalresearch[CommonDataSet].
Retrievedfrom
http://www.luc.edu/media/lucedu/institutionalresearch/cds/CDS%2020152016.pdf
Newcomer,K.E.,&Triplett,T.(2010).Usingsurveys.InWholey,J.S.,Hatry,H.P.,&
Newcomer,K.E.(Eds.)(2010).
Handbookofpracticalprogramevaluation(Third
Edition)
.SanFrancisco,CA:JosseyBass

MEETINGTHELLCOUTCOMES

30

AppendixA
LeadershipforSocialChangeLearningCommunity
MISSION
TheLeadershipforSocialChangelearningcommunityishometoadiversegroupof
studentswhoshareapassionforexploringtheirpersonalidentitiesandvalues,while
increasingtheircapacitytoworkcollaborativelywithotherstocontributetopositive
socialchange.

VISION
StudentsparticipatingintheLeadershipforSocialChangelearningcommunitywillbe
providedwithopportunitiestoincreasetheirknowledgeandskillsrelatedtoleadership
development,andtoauthenticallyreflectuponmultipleaspectsofidentity,andpersonal
values.Communitymemberswillbeencouragedtocommittothedevelopmentof
culturalcompetence,andtoenhanceskillstosupporteffectivecollaborationwithingroup
andteamsettings.Participantswilldevelopcompetenciesandanincreasedsenseof
selfefficacytosupporttheirengagementasagentsforpositivesocialchange.

LEARNINGOUTCOMESFORALLLEARNINGCOMMUNITIES
AsaresultoftheirparticipationinaLearningCommunity,andwhencomparedto
nonLCresidentialstudents,LCstudentswill:

Persistfromsemestertosemesterandtograduationathigherrates

AchievehigherGPAs

AdjusttoLUCsociallymorequickly

Developsupportiverelationshipswithfaculty
ReportanincreasedsenseofconnectednesstoLUC
DevelopanincreasedunderstandingoftheLCsthemeandthewaystoengage
thatthemeoncampus
.

LFSCGOALS
AfterayearasamemberoftheLeadershipforSocialChangelearningcommunity,
studentswill:

Developasenseofcommunityandconnectionwitheachother,aswellaswith
thementors,staff,andfacultysupportingthelearningcommunity
Feelempoweredastheyengageoncampusandbeyond
Gainknowledge,skills,andattitudesthatcontributetotheirdevelopmentas
sociallyresponsibleleadersworkingcollaborativelyforpositivechange

MEETINGTHELLCOUTCOMES

31

AppendixA(cont.)

Demonstratethattheyhaveachievedacademicsuccess,andbeabletoapply
leadershipprinciplesintheiracademicpursuits(leadershipisinterdisciplinaryin
nature)

LEARNINGOUTCOMES
Asaresultofbeinganactivemember*oftheLeadershipforSocialChangelearning
communityforfirstyearstudents,studentswillbeableto:

Describeatleast4ofthedifferentprograms,services,andopportunitieson
campus(andbeyond)wheretheycanconnectandengageintheleadership
process
Identifyatleastoneareaoncampusinwhichtheywereactivelyinvolvedintheir
firstyearatLUC,anddescribeplansforinvolvementinatleastoneareason
campusinthenextacademicyear
Nameanddescribeatleast4ofthevaluesoftheSocialChangeModelof
leadershipandrelatethevaluestopersonalexperience(s)
Engageinreflectionandarticulategrowthinselfawareness
Articulatewaysinwhichtheyhavelearnedtomoreeffectivelyworkwithothers
inaccomplishingagoal
Articulatehowtheyhavegrownintheareaofculturalcompetenceandingaining
adeeperunderstandingofmultipleperspectives
Articulateoneormorelocal,national,orglobalissue(s)aboutwhichtheyare
passionate
Developandarticulateasenseofcommunitywithpeers
Nameatleast3stafforfacultywhomtheyconsidertobeasourceofsupportand
information
*ActivemembersareregularlyengagedinmonthlyLCprograms,andparticipatein
manyoftheadditionalprogramsfacilitatedbytheLCAsand/orRAsonthefloor

Lastedited:June26,2014

MEETINGTHELLCOUTCOMES

32

AppendixB

MEETINGTHELLCOUTCOMES

33

AppendixC

LoyolaUniversityChicagosResidenceLife

Loyola University Chicagos Residence Life requests your help in evaluating the experience of students in the Leadership
for Social Change Learning Community (LLC). The purpose of this survey is to examine the extent to which LLC first year
students are acquiring specific competencies related to the learning outcomes set by Residence Life . The scale below will
range from 1= Not Confident, 2=Somewhat Confident, 3=Confident, to 4= Very Confident.
For the purposes of this survey we ask you to reference Allen Johnsons definition of social change-Social Change is any
alteration in the cultural, structural, population, or ecological characteristics of a social system such as a society. (The
Blackwell Dictionary of Sociology 2000: 285)
Thank you for your time.

Practical Application

I am confident in my ability to:

1.

Seek out leadership opportunities at Loyola University Chicago

Not Confident

2.

Very Confident

Relate personal experiences to social change

Not Confident

3.

Very Confident

Engage in reflection related to my own experience with leadership

Not Confident

4.

4
Very Confident

Recognize my growth in becoming more self-aware of my actions in the community

1
Not Confident

4
Very Confident

MEETINGTHELLCOUTCOMES

5.

34

Effectively work with others to accomplish group projects

Not Confident

Cultural Competency
6.

Very Confident

I am confident in my understanding of the term cultural competency

Not Confident

7.

Very Confident

I consistently seek to understand multiple perspectives and identities different than my own

Not Confident

Very Confident

8.
I can articulate one or more social issues (local, national or global) that I am passionate
about
1

Not Confident

9.

Very Confident

I can apply knowledge of multiple perspectives and identities to my leadership style

Not Confident

10.

Very Confident

I understand how my identities can affect social change

1
Not Confident

4
Very Confident

MEETINGTHELLCOUTCOMES

35

Curriculum & Co-curriculum

11.
Programs (events led by your Learning Community Assistant) contributed to my
understanding of social change
1

Disagree

Agree

12.
Academic curriculum (UNIV 101, Political Science 101, Philosophy 130, Sociology 101)
contributed to my understanding of social change
1

Disagree

4
Agree

13.

List Loyola programs or opportunities where you can engage with the leadership process:

Demographics

Gender:

Male

Female

Not Listed______________
(Please write-in response if not listed)

Race

White

Native American or American Indian

Black or African American

Hispanic or Latino

Asian/ Pacific Islander

Not Listed________________
(Please write-in response if not listed)

Thank you very much for taking the time to complete this survey. Your feedback is valued and very much appreciated!

MEETINGTHELLCOUTCOMES

36

AppendixD

LoyolaUniversityChicagosResidenceLife

Loyola University Chicagos Residence Life requests your help in evaluating the experience of students in the Leadership
for Social Change Learning Community (LLC). The purpose of this survey is to examine the extent to which LLC first year
students are acquiring specific competencies related to the learning outcomes set by Residence Life . Please complete the
following survey based on your experience in the Leadership Learning Community. The scale below will range from 1=
Not Confident, 2=Somewhat Confident, 3=Confident, to 4= Very Confident.
For the purposes of this survey we ask you to reference Allen Johnsons definition of social change-Social Change is any
alteration in the cultural, structural, population, or ecological characteristics of a social system such as a society. (The
Blackwell Dictionary of Sociology 2000: 285)
Thank you for your time.

Practical Application

Based on participation in the Leadership Learning Community, I am confident in my ability to:

1.

Seek out leadership opportunities at Loyola University Chicago

Not Confident

2.

Very Confident

Relate personal experiences to social change

Not Confident

3.

Very Confident

Engage in reflection related to my own experience with leadership

1
Not Confident

4
Very Confident

MEETINGTHELLCOUTCOMES

4.

37

Recognize my growth in becoming more self-aware of my actions in the community

Not Confident

5.

Very Confident

Effectively work with others to accomplish group projects

Not Confident

Very Confident

Based on participation in the Leadership Learning Community, I am confident in my ability to:

Cultural Competency
6.

I am confident in my understanding of the term cultural competency

Not Confident

7.

Very Confident

I consistent seek to understand multiple perspectives and identities different than my own

Not Confident

8.

Very Confident

I can articulate one or more issues (local, national or global) that I am passionate about

Not Confident

9.

Very Confident

I can apply knowledge of multiple perspectives and identities to my leadership style

Not Confident

10.

Very Confident

I understand how my identities can affect social change

1
Not Confident

Confident

4
Very Confident

MEETINGTHELLCOUTCOMES

38

Based on participation in the Leadership Learning Community, I am confident in my ability to:

Curriculum & Co-Curriculum

11.
Programs (events led by Learning Community Assistant) contributed to my understanding
of social change
1

Disagree

Agree

12.
Academic curriculum (UNIV 101, Political Science 101, Philosophy 130, Sociology 101)
contributed to my understanding of social change
1

Disagree

4
Agree

13. List Loyola programs or opportunities where you can engage with the leadership process:

Thank you very much for taking the time to complete this survey. Your feedback is valued and very much appreciated!

MEETINGTHELLCOUTCOMES

39

APPENDIXE

FocusGroupProtocol
Introduction
Hello,andthankyouforattendingthisfocusgroup,mynameis_____________andIwillbe
moderatingtheconversationtoday.Thepurposeofthisfocusgroupistofindoutmoreabout
yourexperiencesintheLeadershipLearningCommunity.Wealsohaveassistantmoderator,
________,tohelptakenotes.StudentLeadershipDevelopmentandResidenceLifearelooking
tofindoutmoreaboutyourexperiencesintheLLCinordertoensurethatthelearningoutcomes
arebeingsufficientlymet.YourresponseswillbeusedtohelpimprovetheLLCinfutureyears.

Inthenexthourandfifteenminutes,IwillaskaseriesofquestionsrelatedtoyourLLC
experience.Thisismeanttobeaninformalconversationso,feelfreetospeakupatanypoint,
orlistenwhenothersarespeaking.Youarenotrequiredtoanswereveryquestionorraiseyour
handtospeak.Ifyouhaveadifferentopinionorperspectivewewouldencourageyouto
becomeapartoftheconversation,soweunderstandtheviewpointsofasmanypeopleas
possible.Therearenorightorwronganswerstoanyofthesequestions.Althoughtheassistant
moderatorwillbetakingnotes,wealsoplantoaudiorecordtheconversationtoensurethatwe
accuratelycaptureeverything.Allnameswillremainconfidentialinthefinalreportand
similarly,weaskthatyourefrainfromsharinganyinformationorresponsessharedduringthe
focusgroup.Arethereanyquestions?

Atthispoint,pleasereadovertheinformedconsentformyoureceivedandsignyournameatthe
bottom.Iwillcollectthemwheneveryoneisready.Arethereanylastquestionsbeforewe
begin?Wewillnowturnontherecorder.

ThisfocusgroupisbeingconductedfortheLeadershipLLCevaluationon[date]by[moderator]
and[assistantmoderator].Thestarttimeis[time].

DiscussionQuestions
GroupIntroductions:
1.
Irealizethatmanyofyouhavebeenlivingaroundeachotherforthelastyearbutlets
goaroundtheroomandintroduceourselvesbysayingourfirstname,major,andafunfactabout
yourself.
2.
WhatinitiallydrewyoutotheLeadershipLearningCommunity?
a.
Didanythingjumpoutwhenyoureadaboutitinthedescription?
b.
Whatexperiencesdidyouhopetohave?
3.
InwhatwayshaveyourexpectationsofjoiningtheLLCbeenmet?
a.
Couldyougiveanexample?

MEETINGTHELLCOUTCOMES

40

4.
InwhatwayshaveyourexpectationsofbeingintheLLCnotbeenmet?
a.
Couldyougiveanexample?
b.
WouldyoudescribeyourexperienceintheLLCaspositive,negative,orneutraland
why?
5.
HaveyouhadanopportunitytopracticeleadershipoutsideoftheLLCifso,howhas
yourexperiencewithintheLLCinfluencedthis?
a.
HowwouldyouplantouseyourexperiencesfromtheLLCinfutureleadershiprolesor
settings?
6.
Whatdoesittaketoaccomplishsocialchange?
a.
Howdoyouridentitiescomeintoplay?
b.
Howdootherpeoplesidentitiescomeintoplay?
7.
Howwouldyoudefineculturalcompetency?
a.
WhatexperiencesinLCChaveyoucontributedtoyourunderstandingofculturalcompetency?
b.

WhatexperiencesinLCChaveyoucontributedtoyourunderstandingofsocialchange?

8.
HowsatisfiedwereyouwiththecoursesyoutookintheLLC?
a.
Whatspecificallyaboutthecontentyoulearneddidyoulikeordislike?
9.
Howwouldyoudefineleadership?
10.

Isthereanythingthatyouwouldliketosharethatyouhaventhadtheopportunityto?

ThoseareallthequestionsIhaveforyouarethereanyquestionsyouhaveforus?

Thankyouforyouagainforyourparticipationintodaysdiscussion.Yourinputandparticipationtodayis
greatlyappreciated.Theinformationyouofferedwithbetakenintoconsiderationineffortstoimprove
academicadvisingservicesintheCASDeansOffice.Pleasefeelfreetocontactmewithanyquestions.

APPENDIXF

MEETINGTHELLCOUTCOMES

41

FocusGroupConsentForm
ProjectTitle
:LeadershipLearningCommunity
Evaluator
:EricRuelle/DeNeatriaRobinson
Introduction
:
YouareabouttotakepartinafocusgrouptohelptheLeadershipLearningCommunityat
LoyolaUniversityChicagogainabetterunderstandingofyourexperiencesandwhetherthe
initialoutcomesfortheLLCarebeingmet.Pleasereadthroughthefollowinginformation
carefully.Ifyouhaveanyquestionsregardingtheinformationbelow,pleaseaskthemoderator
priortodecidingwhethertoparticipate.
Procedures
:
Onceyouagreetoparticipateinthefocusgroup,youwillbeaskedaseriesofquestions
pertainingtoyourexperiencesintheLeadershipLearningCommunity,thepositivesand
negatives,aswellasthemajortakeawaysoftheprogram.Thereisnorightorwronganswerand
weaskthatyourespondhonestandopenly.Pleaseknowthatyouarenotrequiredtorespondto
anyofthequestionsandshouldonlyrespondbasedonyourcomfortlevel.
Confidentiality
:
Byparticipatinginthefocusgroup,yournamewillbeomittedfromanyresponsesyouprovide.
Anyinformationprovidedwillbecompiledinareportandbasicthemeswillbesharedwithyou
priortofinalizingit.ThecompiledreportwillbesharedwithMarciWhalton,AssistantDirector
ofResidenceLife,andShannonHowes,DirectorofStudentLeadershipDevelopment,atLoyola
UniversityChicagoinordertohelpmakeimprovementsinreachingthelearningoutcomesfor
studentsintheLLCmovingforward.
VoluntaryParticipation
:
Participationinthisfocusgroupisvoluntary.Atanypointinthefocusgroup,youarefreeto
withdrawfromparticipation.Pleaseknowthatyouarenotrequiredtostayfortheentiretime.
StatementofConsent
Bysigningbelow,youareagreeingtothetermsandconditionsabove,aswellasreleasingany
informationprovided

duringthefocusgroup.Additionally,youhavehadachancetoaddressany
questions,comments,andconcernsandagreetoparticipateinthefollowingfocusgroup.Acopy
ofthisdocumentwillbeprovidedforyourrecords.
_______________________________________
Participant'sSignature:
Date_____________
EvaluatorsSignature:
Date_____________

MEETINGTHELLCOUTCOMES

42

APPENDIXG
TIMELINE
Task

2016

JUL

AUG

PresentEvaluation
PlantoResidence
Life&Student
Leadership
Development

PretestSurvey

DataAnalysis&
CodingofPretest
Survey

SEP

2017

OCT

NOV

DEC

JAN

FEB

MAR

APR

MAY

JUN

JUL

AUG

SolicitParticipants
fortheFocusGroup

FocusGroup

Transcribe
Recordings

PosttestSurvey

DataAnalysis&
CodingFocusGroup

DataAnalysis&
CodingPosttest
Survey

Presentationof
Findings

DevelopNextSteps

ImplementNext
Steps

MEETINGTHELLCOUTCOMES

43

APPENDIXH
Budget
Activity

Item

Cost/Item

Quantity

Total

SurveyAdministration

PaperPreSurveys

.07

70

$5

PaperPostSurveys

.07

70

$5

Administrators

Free

Free

FocusGroups

DigitalRecorder(rent
fromLUC)

Free

Free

DinnerforFocusGroups

$10/person

40

$400

ConsentForms

.07

40

$3

FocusGroupModerator

Free

2Evaluators

Pens(boxof12)

$2

$2

Analysis

Transcribers

Free

2Evaluators

Free

Coders

Free

2Evaluators+1

Free

TotalCost: $415

MEETINGTHELLCOUTCOMES

44

APPENDIXI