Mageary and Randolph Elps431 Final Evaluation Plan

Running head: BROTHERS FOR EXCELLENCE EVALUATION PLAN 1
Brothers for Excellence Evaluation Plan
Colin Mageary and Patrick Randolph
Loyola University Chicago

BROTHERS FOR EXCELLENCE EVALUATION PLAN 2
TABLE OF CONTENTS
PROGRAM OVERVIEW ................................................................................................ 3
PROGRAM CONTEXT ......................................................................................................... 3
PROGRAM HISTORY .......................................................................................................... 3
CONCEPTUAL FRAMEWORK .............................................................................................. 4
PROGRAM PURPOSE .......................................................................................................... 5
PROGRAM DESCRIPTION ................................................................................................... 6
STAKEHOLDERS ................................................................................................................ 8
PREVIOUS EVUALUATION ......................................................................................... 9
LOGIC MODEL ............................................................................................................. 11
GENERAL ASSESSMENT APPROACH .................................................................... 13
QUANTITATIVE APPROACH.................................................................................... 15
SURVEY POPULATION ..................................................................................................... 15
SURVEY & EVALUATION DESIGN ................................................................................... 15
SURVEY INSTRUMENT ..................................................................................................... 17
PILOT DESIGN ................................................................................................................. 17
STATISTICAL ANALYSIS .................................................................................................. 17
FINAL REPORT ................................................................................................................ 18
QUALITATIVE APPROACH ...................................................................................... 19
QUALITATIVE RATIONALE .............................................................................................. 19
PARTICIPANT POPULATION ............................................................................................. 21
INSTRUMENT................................................................................................................... 21
IMPLEMENTATION .......................................................................................................... 22
DATA ANALYSIS ............................................................................................................. 23
PRESENTATION OF RESULTS ........................................................................................... 25
LIMITATIONS ............................................................................................................... 25
TIMELINE ...................................................................................................................... 26
BUDGET .......................................................................................................................... 27
NEXT STEPS .................................................................................................................. 28
REFERENCE .................................................................................................................. 29
APPENDICES ................................................................................................................. 31
A. BROTHERS FOR EXCELLENCE CONTRACT ................................................................. 31
B. LOGIC MODEL .......................................................................................................... 32
C. QUANTITATIVE SURVEY INSTRUMENT ...................................................................... 33
D. QUANTITATIVE SURVEY MATRIX ............................................................................ 57
E. EMAIL TEMPLATES ................................................................................................... 62
F. INFORMED CONSENT FORM ...................................................................................... 63
G. DEMOGRAPHIC VARIABLE FORM .............................................................................. 64
H. INTERVIEW PROTOCOL ............................................................................................. 65
I. TIMELINE .................................................................................................................. 68
J. FINAL POWERPOINT PRESENTATION ........................................................................ 69
Brothers for Excellence Evaluation Plan
Program Context
Brothers for Excellence is a one-year program for first-year and transfer men of color at
Loyola University Chicago (LUC). Offered through the Department of Student Diversity and
Multicultural Affairs (SDMA), the program connects these students to two mentors, one an
upperclass peer and one faculty or staff member, both of whom identify as men of color.
The Department of SDMA consists of five full-time professional staff and two graduate
assistants. Quortne Hutchings, a Program Coordinator, oversees the program, including
supervision of the eight Peer Mentors, with occasional support from Anthony Sis, a graduate
assistant (Hutchings, personal communication, September 14, 2016). Hutchings and another
Program Coordinator in the department have been in their roles for less than a year, though the
department has had relatively consistent staffing otherwise in the past few years. Their office is
located on the first floor of the Damen Student Center on the Lake Shore Campus. Most of the
program takes place on the Lake Shore Campus and surrounding area, although some initiatives
occur off-campus in Chicago, Illinois and at the Loyola University Retreat and Ecology Campus
in Woodstock, Illinois (Hutchings, personal communication, September 14, 2016). Politically,
SDMA falls within the Student Life and Engagement section of the Division of Student
Development at Loyola, thereby reporting to the Dean of Students, K.C. Mmeje. Brothers for
Excellence specifically has an annual budget of $10,500, $4500 of which is designated
specifically for an annual retreat.
Program History
Brothers for Excellence began five years ago as a space for men of color to find support
in the different facets of college life, including academic support, social support, leadership
opportunities, and a group of people to discuss common experiences as men of color (Hutchings,
personal communication, September 14, 2016). Originally titled Men of Color Initiatives, the
program grew out of the Jesuit value cura personalis (care for the whole person), and in the
early years of the program, mostly black identified men participated. After two to three years of
this trend, former Program Coordinator Miguel Macias saw a need to reach out to all men of
color on the Loyola University Chicago campus. The program rebranded itself as Brothers for
Excellence and began to encourage all men of color to become more involved than they
previously had been (Hutchings, personal communication, September 14, 2016). The program
has been running for five years now with no end date in sight.
Conceptual Framework
According to Quaye, Griffin, and Museus (2015), students of color at predominantly
white institutions tend to participate less with the campus than white students. Students of color
may also feel a disconnect to the Eurocentric presumptions and considerations promoted by
many predominantly white institutions, leading to feelings of detachment, alienation, and
disengagement (Quaye, Griffin, & Museus, 2015). Logically then, these mindsets may influence
ones academic record, and subsequently, retention and graduation rates. Additionally,
American society encourages men to conform to narrowly-constructed and stereotypical
masculine behavioral norms, which discourage or detract their engagement in behaviors and
activities that lead to desirable outcomes (Lester & Harris III, 2015, p. 153). To that end, male
students tend to not be involved with their campus, abuse substances, and engage in misconduct
(Lester & Harris III, 2015). On an intersectional level, men of color then face many difficulties
when entering the college environment.

Much of the existing literature focuses on black men in particular. Men of color are often
marginalized based on their racial identities, and while this gap is significant, program designers
and evaluators need to interpret the applicability of findings to other men of color to be more
inclusive and ground their actions in the literature.
Several works of literature, however, assert that engagement with the campus can assist
black men in their higher education journey in the face of racial oppression. Dancy II (2011)
found that black men at predominantly white institutions often feel pressures to conform to a
hyper-masculinity construct and representations of black men in the media, but that mentorship
can provide spaces to discuss and manage those challenges. To that end, Strayhorn (2008)
concluded that close relationships with peers, faculty, and staff significantly affect black mens
satisfaction with their college experience. Moreover, Strayhorn (2010) determined that campus
involvements led to both personal development and higher academic achievements for black
men. As such, mentorship and engagement on various levels can assist black men, and by
extension, other men of color, in coping with many issues they encounter on college campuses,
especially at predominantly white institutions.
Program Purpose
The purpose of Brothers for Excellence is to combat these types of issues men of color
typically encounter when they enter the college environment (Hutchings, personal
communication, September 14, 2016). Firstly, the program aspires to engage students with the
Loyola University Chicago campus, including involvement and developing awareness of
resources. Secondly, it hopes to offer academic support to a student population that may not
seek support. Finally, Brothers for Excellence provides a brave space for men of color to engage
in critical dialogues about several issues, such as toxic masculinity and identity
development. According to Hutchings, men of color have typically struggled in these areas, but
providing this mentorship opportunity can boost participants confidence and ability to succeed
at Loyola University Chicago (personal communication, September 14, 2016). Brothers for
Excellence strives to promote engagement, mentorship, and camaraderie among participants to
counteract the difficulties these two identities often encounter when coming to a predominantly
white institution, thereby bridging the gap and creating more opportunities for men of color to
succeed at Loyola. That being said, Hutchings does not operate from a deficit lens, but rather
assumes that all participants have cultural capital to contribute to a positive program dynamic
(personal communication, September 14, 2016).
The above information regarding the goals of the program was synthesized from the
Brothers for Excellence website, the programs contract signed by all participants (see Appendix
A), and an interview with Hutchings (Brothers for Excellence (B4E), n.d.; Hutchings, personal
communication, September 14, 2016). That being said, there are no concrete learning outcomes
in a centralized location for the program. Although overarching goals for Brothers for
Excellence can be articulated, a lack of recurring, exact goals is a concern to raise when
developing this evaluation plan. As such, outcomes for the logic model (see Appendix B) were
extracted from the above synthesis.
Program Description
The invitation to apply for the program goes out to every self-identified man of color
accepted to Loyola University Chicago (Hutchings, personal communication, September 14,
2016). Due to this, men who participate in the program have a wide range of backgrounds and
identities. As mentioned, each student receives one Peer Mentor and one faculty or staff mentor
(Hutchings, personal communication, September 14, 2016). Peer Mentors, who all have
previously participated in the program, meet with their mentee at least once a month to develop
relationships and offer guidance and support. Hutchings requires the Peer Mentors to file weekly
reports on their interactions, including discussion topics, interpersonal dynamics, and how often
the Peer Mentor has reached out. Additionally, Peer Mentors hold weekly office hours in the
office space of SDMA and should be in contact with their mentees through text messaging,
email, phone conversations, and/or social media. Faculty and staff mentors do not necessarily
need to meet on a regular basis with their mentees, but that is encouraged by Hutchings.
In addition to one on one conversations, the department offers large-scale programming
throughout the semester that Peer Mentors, faculty and staff mentors, and mentees are all
required to attend per a contract signed at the beginning of the semester (Hutchings, personal
communication, September 14, 2016). All together, there are 14 programs, each of which is
designed to help facilitate different facets of learning (see Appendix A for list of events).
In the future, Hutchings envisions the program growing to become more of a community
than a one-year program (Hutchings, personal communication, September 14, 2016). A
challenge he has encountered is that very few upperclassmen and alumni continue their
involvement with Brothers for Excellence, as the program was not built to keep participants
active or involved after its conclusion. Because of this, there is also little data on what
participants do following the completion of the program, such as academic achievement,
graduation rate, job placement, and campus involvement. Hutchings would also like to
incorporate more campus partners into the program and more professional development
opportunities for the men in the program (personal communication, September 14, 2016). A
large component of that would be helping students attend and eventually present at different
professional conferences, thereby highlighting how the students have grown and benefited from
the program.
Stakeholders
Direct stakeholders in the program are the first-year and transfer students who participate
in Brothers for Excellence, as they receive direct benefit from the programs intended goals and
outcomes. Indirectly, the Peer Mentors and faculty and staff mentors also benefit from the
leadership skills developed and opportunities to enhance their involvement on Loyolas
campus. Additionally, SDMA as a unit serves as a key stakeholder in that Brothers for
Excellence is a marquee program offered by the department and its staff want to see the program
succeed to continue offering helpful opportunities and resources for marginalized
students. Moreover, campus partners, including Residence Life, First and Second Year
Advising, and the Office of Student Conduct and Conflict Resolution, among others, hope for
SDMA to succeed to continue utilizing them for referrals and resources (Hutchings, personal
communication, September 14, 2016).
Critical decisions for the program are on the horizon, revolving around participation,
engagement, and funding (Hutchings, personal communication, September 14, 2016). Hutchings
wants to begin looking to academics, attendance, job placement, and student growth to
demonstrate the necessity for Brothers for Excellence. To justify continued funding for the
program, quantitative data surrounding grade point average, job placement rates, retention rates,
and graduation rates could prove useful to provide prominent stakeholders at the University.
As those presenting this evaluation plan, we consider ourselves indirect stakeholders as
well. Both of us were interested in the Brothers for Excellence program due to its nature of
masculinity development. We both were involved with masculinity initiatives and trainings as
undergraduate students and have continued that interest as graduate students in the Higher
Education masters program at Loyola. With the expansion of our understanding of power and
privilege, we were also interested in the intersectionality that comes with identifying as both a
man and a person of color, as the Brothers for Excellence program provides spaces and resources
for men from all different backgrounds to, be a man, and be a man of color (Hutchings,
personal communication, September 14, 2016). That being said, we both identify as white men,
but the opportunity to develop this evaluation plan and provide a tool for the program to advance
in a positive direction excites us both. We hope for this program to succeed and continue to
prosper as a result of our efforts.
Previous Evaluation
Regarding previous evaluation efforts done by SDMA, there are few records of
assessment for Brothers for Excellence. Other than the Peer Mentor applications, the only data
on file are the Peer Mentors weekly reports, which Hutchings reads to ensure mentees are being
engaged and provided the necessary resources (Hutchings, personal communication, September
14, 2016). However, assessment is an area Hutchings desires to explore in the future to evaluate
the experiences and successes the mentees are having as a result of the program.
Two fluctuations in resources may affect the program and distort the assessment
process. The first matter is that Hutchings is new to his role in SDMA. While this is not
Hutchings first time as a professional in the field of student affairs, he is still learning the history
of the program while also striving to improve upon previous successes (Hutchings, personal
communication, September 14, 2016). To that end, the lack of previous assessment measures
Hutchings possesses has led us to have less knowledge and context about the
program. Secondly, the varying levels of commitment on the part of participants could eschew
data. While the program has the potential to benefit students, it does take time and dedication to
do so. Some mentees are asked to leave the program due to not attending sessions or responding
to their mentors (Hutchings, personal communication, September 14, 2016). Additionally, Peer
Mentors need to dedicate time and energy to their role to support their mentees and in turn make
the program successful in achieving its goals. These are assumptions noted in the logic model
(see Appendix B), but are still important considerations. Moreover, while Hutchings and his
staff are committed to the program, they also have other programming they oversee, which may
divide their attention.
Additionally, while budgetary resources have not fluctuated, it could be a significant
factor in the future and could drive assessment needs. Within the field of higher education,
finances drive ones ability to program, and while this programs budget has remained steady at
$10,500, finances are never guaranteed in the future. As such, assessment measures are
necessary to evaluate the efficiency and effectiveness of Brothers for Excellence to continue
legitimizing it. For SDMA to continue allocating resources to the program in the future, the
Department needs to provide this sort of data. We plan to provide this document to Hutchings in
order for him to collect and interpret data to improve upon current practices, as well as provide
the program with more credibility.
We wish to analyze and uncover the benefits men of color experience by engaging in this
program. It is our goal in this plan to outline data collection methods that will support Hutchings
and his staff in producing both quantitative and qualitative data. As stated, men of color
typically could benefit from a mentorship program like Brothers for Excellence, and as such, we
hope, with the evaluation method proposed in this plan, to answer the following overarching
questions:
How engaged are men of color with their campus as a result of this program?
How has this program affected men of colors grade point averages, retention rates,
willingness to participate as alumni, and professional development?
How has the program affected identity development as men, people of color, and men of
color?
We will gather data in an attempt to answer these three general questions. To address the
questions noted above, this evaluation will be formative in purpose and focus on both the process
and outcomes (see General Assessment Approach for more information).
The above questions, purpose, and focus were all selected based on the desire to provide
SDMA with data to support the credibility of this program. The questions selected are only three
goals of the program, but we selected these in particular because we believed they best
encompassed the central purpose of Brothers for Excellence. This assessment plan is also
formative, not summative, because it is focused on program improvement and evaluates the
merit of the programs intended outcomes. Additionally, we wanted to focus on both the
outcomes of the program (in terms of enrollment record, identity development, and campus
engagement) for the mentees benefit, as well as the process by which they reach those outcomes
to provide Hutchings with as much data as possible to analyze and assimilate into practice.
Logic Model
A logic model was designed to map out the Brothers for Excellence program. As noted
by the Kellogg Foundation (2004), a logic model is a visual representation of a programs
components, the operations of a program, and the potential effects and outcomes of the program.
Components of a logic model include available resources and program activities, which describe
intentional efforts by program coordinators, and outputs, outcomes, and impacts, which reflect
hopeful results of the program (Kellogg Foundation, 2004). Logic models are used for
assessment because, Mapping a proposed program helps you visualize and understand how
human and financial investments can contribute to achieving your intended program goals and
can lead to program improvements (Kellogg Foundation, 2004, p. 3). As such, these visual
models can contribute greatly to comprehending all components of a program and how effective
they are at achieving their goals.
As noted in the logic model (Appendix B), the largest resources utilized by Brothers for
Excellence are the effort, time, and support of Program Coordinator Quortne Hutchings, who is
responsible for planning and implementing all 14 of the Brothers for Excellence events,
recruitment and selection of all participants, and supervising the Peer Mentors. It is also
significant to the Logic Model to note that Hutchings is entering his first full year in his role and
will be making large decisions that affect the direction of Brother for Excellence in the near
future.
It is also important to assume that the mentee participants (first-year and transfer men of
color students) are seeking a group for support in academic, social, and professional realms.
This assumption drives the whole program due to the commitment needed from participants to
meet the learning outcomes of Brothers for Excellence. The commitment needed to the program
then makes the assumption that all participants attend all agreed upon programming. A final
important assumption is that participants will form positive connections with their mentors,
which is critical given that the program is predicated on the formation of these didactic
relationships. Acknowledging assumptions of the program highlights properties that could have
significant influences on the programs success. In turn, they could effect how Hutchings
decides to alter the program in future years.

Much of the mentees intended development occurs as a result of numerous Brothers for
Excellence events scheduled throughout the year. Programming includes social gatherings,
service events, professional development opportunities, a retreat at the Retreat and Ecology
Campus, and consistent one-on-one meetings with the assigned mentors. It is through these
events that participants in Brothers for Excellence are receiving the support and tools to be
successful and thrive at Loyola University Chicago. What is unknown, however, is how
successful these events are at accomplishing the intended outcomes. With no previous
assessment measures, the following evaluation questions arose: To what extent do Brothers for
Excellence participants complete the intended learning outcomes? What processes are
successful in fostering these learning outcomes?
General Assessment Approach
As stated above, the assessment approach most applicable to this evaluation plan is a
combination of process and outcome assessment. With no previous assessment having been
conducted, we felt it was necessary to evaluate as much of the program as possible. We will be
employing both the process-based approach to assessment, namely focusing on the operations
and procedures of a program, as well as the outcomes-based approach, meaning assessing the
intended results of the program (Keeling, Wall, Underhile, & Dungy, 2008) We feel that the
combination of processes and outcomes will allow us to see the effectiveness of the programs
intended outcomes, as well as the extent to which students reach those outcomes through
Hutchings current practices. We will also be using formative assessment practices, namely,
providing feedback that can be used to modify, shape, and improve the program (Banta &
Palomba, 2015, p. 19). Assessing the outcomes and processes of the program from a formative
standpoint will allow Hutchings and his staff to determine how they need to adjust their
programming practices to better support men of color. In doing so, data can be analyzed and
assimilated into practice to improve the program and justify its merit at the University.
Additionally, while the program is still fairly new, entering its fifth year of operation, it is in a
pivotal place to determine its future success. The more data we are able to provide Hutchings,
the better he will be able to advance the program and justify its continuation through empirical
analysis.
Potential strengths to this assessment approach are the commitment to program
improvement and the wide array of data. The combination of approaches will provide a large
pool of data. Analyzing these data to determine not only if outcomes are achieved, but also how
they are achieved, can lead to better informed student affairs practice. Subsequent adjustments
to the tactics utilized to accomplish the programs goals can increase the effectiveness of
achieving higher rates of student engagement, enrollment numbers, and identity development.
However, while the extensive data are an advantage, they will lead to a more difficult collection
process due to the need to design more complex instruments. Moreover, these data will be
largely grounded in participant responses to that more complex instrument, which may lead to
lower response rates. Additionally, determining if outcomes were met is subjective to the
individual mentee and may vary, which is significant to remember.
As stated earlier, there are three specific outcomes the evaluator will measure. Firstly,
we hope to determine the extent to which mentees in the program became more engaged on the
Loyola University Chicago campus, including their campus involvement and leadership
positions obtained. Secondly, we aspire to measure mentees identity development in terms of
their male and person of color identities, as well as the salience of those identities intersecting
with one another. Finally, data will be collected to assess mentees enrollment status, including
grade point average, first- to second-year retention rates, alumni participation, and professional
development for their future goals. Regarding processes, mentees will evaluate the extent to
which their relationship with their Peer Mentor, their relationship with their faculty/staff mentor,
and the large-group programming affects their ability to achieve the three intended outcomes
outlined above. Collecting and analyzing data in both of these areas is significant because they
demonstrate how effective this departments current practices are and how extensively goals are
being met. In doing so, they relate back to the larger goal of this evaluation initiative of
adjusting the programs practices based on data to better align with Brothers for Excellences
intended outcomes.
Quantitative Approach
Survey Population
This survey will be administered as a post-test measure to all mentees of the Brothers for
Excellence program, namely first-year or transfer students that identify as men of
color. Hutchings will administer the survey himself. Specifically, this survey this will be a
census because all participants will be given the opportunity to participate, which is warranted
given the small population of approximately 40 to 45 mentees. Convenience and census
sampling were chosen because we hope to gather as much data as possible within this relatively
small population of approximately 45 students. As such, probability sampling strategies would
not be beneficial and limit data gathered.
Survey and Evaluation Design
Our survey will be administered through a Google form. The form was created with
eight different sections, all of which were designed to assess a particular outcome or process.
We will have all mentees in Brothers for Excellence take our survey during a scheduled meeting,
namely the final meeting of the year at the end of the spring semester. Our participants will have
experienced the entire year-long program and the questions were written with this in mind.
There will be no incentive to complete the survey because it will be a requirement of the
program in order to create more informed practices that can better support future participants.
Email reminders that the survey will be occurring will be sent in advance. Recognizing that
some participants may have to miss the scheduled survey time, the link will be distributed to
provide the opportunity to take the survey on their own time. With this in mind, we hope to
receive survey responses from at least 90% of B4E participants.
We will be using a cross-sectional research design during our data collection to collect
data at a specific point in time. We selected this design because we are relying on students
personal responses to determine the extent to which outcomes were achieved and processes were
effective. Because such development has the potential to be unique to each individual,
comparison of growth before and after the program, as in a longitudinal design, would not
provide much useful information for measuring processes and outcomes. Conversely, a cross-
sectional design would save evaluators time by measuring subjective development at a single
point in time. Additionally, as Banta and Palomba (2015) noted, participant dropout presents a
challenge to longitudinal designs, and Hutchings also noted that this has been an issue with B4E
in the past (personal communication, September 14, 2016). As such, we selected a cross-
sectional design and questions are designed to gather data about the entire year in Brothers for
Excellence and will be collected at the end of the academic year.
The purpose of the evaluation is to improve the programs processes and its ability to
complete the programs learning outcomes. Having never collected data on the program before,
the survey will create a great starting point for Quortne Hutchings to begin assessing Brothers for
Excellence. With high response rates and intentionally constructed questions, we feel this survey
can gather enough data to make decisions for the improvement of the program.
Survey Instrument
The quantitative survey instrument (see Appendices B and C) consists of 82 questions,
with the majority utilizing ordinal Likert scales ranging from Strongly Disagree (1) to
Strongly Agree (5). Additionally, several questions utilize nominal measures and three
questions within the demographics section are open-ended. The survey should take an estimated
30 minutes to complete. The first three sections of the instrument relate to the programs
intended outcomes we selected to evaluate - Engagement with the LUC Campus, Enrollment
Development, and Identity Development. The next three sections address three processes to be
evaluated - Peer Mentor Relationship, Faculty/Staff Relationship, and Brothers for Excellence
Programs, followed by a short section on participant demographics. Hutchings, as the Program
Coordinator for Mens Initiatives, will be listed as the primary contact.
Pilot Design
Given the little existing data on the program and the small population of students to be
sampled, we have decided to not pilot test our survey. We felt as though a pilot test would not
benefit our evaluation because in essence this evaluation is serving as a pilot design for future
evaluations of the program. Based on how well this evaluation is executed, future evaluators can
modify the current instrument to better assess the selected outcomes and processes outlined in
this plan.
Statistical Analysis
For our statistical analyses, we will do both a descriptive analysis and an inferential
analysis. Firstly, we will run descriptive analyses between several outcomes results and related
processes results and utilize Pearson correlations to determine the strength of those relationships.
Doing so will detail how effective processes are in achieving specific program outcomes. For
the sake of time, Pearson correlations will not be done for every variable due to the number of
combinations, but only for those variables that are related to each other as determine by the
evaluators. Secondly, we will run inferential analyses for participants based on race, sexual
orientation, and class standing (i.e. first-year or transfer student) to determine generalities among
each of those three identities. We hope to formulate both a null hypothesis and an alternative
hypothesis for the program rooted in achieving the designated outcomes using the designated
processes and reject the null hypothesis. To do so, we again will compare the means of these
three identities. Independent samples t-test will be utilized for class standing because we are
comparing the means of two specific groups. Analyses of Variance (ANOVA) will be conducted
to compare the means of racial identities and sexual orientation identities because there are more
than two options for each. SPSS will be utilized to analyze data for both the descriptive and
inferential analyses.
Regarding our analyses, we will be examining change over time within our sample in the
sense that the survey questions relate to mentees sense of growth over time. However, we are
not conducting a pretest-posttest assessment nor is this meant to be a longitudinal study, but
merely surveying if the program assists in the participants growth as outlined in our program
logic model. Additionally, we will not have a treatment group and control group because we are
only analyzing this population and the differences between the three designated identities. To
that end, we are not comparing those who experienced the program with those who did not.
Final Report
In our final report, we will present survey results and highlight the key outcomes
discovered. Bar graphs will be used to convey statistically significant results, positively or
negatively. These will be used due to their aesthetic being relatively easy to understand.
Demographic information will also be displayed in a bar graph in order to see the information in
a quick and simple manner. For descriptive analyses, select Pearson correlations of .60+ will be
included to demonstrate strengths for the program. Conversely, select Pearson correlations of
.00 to .59, as determine by the evaluators, will be included to articulate areas of growth for
Hutchings and the program. For inferential analyses, statistically significant differences will be
included in the bar graphs. For both types of analyses conducted, an appendix will be included
detailing all Pearson correlations, independent t-tests, and ANOVA ran to provide ample
references for Hutchings. These data visualizations will best display the significant survey
responses and will be used to better understand how Brothers for Excellence affects the
mentees. The final report will also highlight key findings from the bar graphs in narrative form
to explain them further, as well as possible implications for future, more effective student affairs
practice.
Qualitative Approach
Qualitative Rationale
For our qualitative approach, we will be using semi-structured interviews with Brothers
for Excellence participants. According to Banta and Palomba (2015), semi-structured interviews
include predetermined questions, but do not necessarily have to be implemented in a strict
order. Moreover, follow-up questions can be utilized to direct conversation based on participant
responses of interest (Banta & Palomba, 2015). Because we hope to examine the lived
experience of these participants within the context of literature on the given population, semi-
structured interviews provide both an outline of topics to explore and flexibility if participants
share different information. Moreover, we elected for semi-structured interviews instead of
focus groups due to concerns for privacy. Adams (2010) noted that semi-structured interviews
may be of value when asking more sensitive questions to participants, as they may be less likely
to answer as honestly if they were in a focus group with others observing them. With the
sensitivity of some topics of interest, including academic performance and identity development,
interviews were utilized to provide a more private, safe space for participants to be candid in
their responses.
With the formative assessment purpose in mind, these interviews will allow evaluators to
gain a more in-depth understanding of the processes and outcomes of the program. Again, the
processes include the Peer Mentor relationship, the faculty/staff mentor relationship, and the
large scale programming, and the outcomes include campus involvement, enrollment and career
factors, and identity development. As Adams (2010) discussed, semi-structured interviews can
support quantitative methods by investigating survey data further following analyses of
results. Open-ended questions can also complement quantitative methods by providing more
data rooted in personal experience (Adams, 2010). As such, these interviews will be
implemented post-survey in order for survey results to inform probing questions and areas of
interest. To that end, while the semi-structured interviews will reflect the same overall
assessment goals as the survey, these qualitative questions will further investigate survey results.
In doing so, we hope that interview findings will either support quantitative findings to make for
more cohesive analyses and/or provide fruitful, differing data to contrast with quantitative
findings. Moreover, interview questions and probes were designed with both the processes and
outcomes under investigation in mind and clearly reflect those topics, just as the quantitative
survey did.
Participant Population
Participants for the interview will be drawn from the larger population of Brothers for
Excellence mentees. As long as they were a participant in B4E, they will be allowed to
participate in the interviews. As Adams (2010) discussed, given the small pool of potential
participants, all will be invited to interview. Potential participants will also volunteer for the
interviews by responding to a question at the end of their original survey. Email templates for
participant recruitment are outlined in Appendix E. Additionally, Appendix F is an informed
consent form participants must sign before interviewing. There are no requirements for GPAs,
class standing, conduct record, demographic, or experience, although participant demographic
information will be collected for subsequent data analysis (see Appendix G for the form).
Instrument
A semi-structured interview will allow the interviewer freedom to explore topics and
points of interest more openly with the participant. It will also give the interviewer the structure
they may need to extract key points of data in the event an interview is not unfolding as
necessary for the study. An interview seemed most appropriate for the participants due to the
complexity and uniqueness of each individuals story. The overarching purpose of the
interviews is to obtain the stories and experiences of participants in Brothers for Excellence. A
semi-structure interview will allow the flexibility and freedom to obtain these experiences
(Adams, 2010). We also wanted to stay away from any group think that could accrue by
utilizing focus groups. We hope participants will speak to their authentic, individual experiences
within the program, but group think may promote participants to not do so in an effort to
conform, which would compromise data validity and have implications for program
improvement.
The interview will take approximately 60 minutes to complete with the possibility of it
going longer or shorter depending on the interviewee. Because all participants will have taken
the survey, the interview questions asked will follow along the same outcomes and processes
assessed in the survey. The interviews will be connected to the survey data to give it a voice and
a deeper meaning. The interview protocol is split into an introductory section, five main
questioning sections, and a concluding section (see Appendix H). Although there is not a
distinct section for each of the three outcomes and three processes, there is significant overlap in
the broad questions selected and probes included for further inquisition. The five categories of
the main questioning sections examine Overall B4E Experience, Academic Experience,
Mentor/Brother Connections, Identity Development, and Campus Engagement, each of which
were chosen to explore the relationships of the outcomes and processes from the perspective of
the participant. As a result, evaluators will be able to gain further insight from quantitative
survey data and understand how processes inform outcomes.
Similarly to the quantitative instrument, the interview questions will not be pilot tested
given the small sample population and little previous data on the program, as the interviews in
themselves are essentially pilot tests for future assessment measures. As such, we felt a pilot test
would not benefit our evaluation of Brothers for Excellence.
Implementation
With Brothers for Excellence concluding in May of each year, interviews will occur at
the beginning of the next academic year, namely late August and the entirety of September. This
scheduling was chosen to allow time for participants to reflect on their experiences within the
program. Additionally, interview participants will have received the full program, permitting a
more complete representation of participant experiences. Interviews will be conducted by
Hutchings and Sis, as they have the most involvement with Brothers for Excellence within
SDMA. Additionally, these two were chosen given their familiarity to participants to provide an
environment of comfort. To that end, Hutchings and Sis will take notes to record participant
responses and interviews will be audio recorded with participant permission. In doing so,
evaluators will have the most accurate records of qualitative data possible with the ability to
verify the accuracy of their written notes. With the ability to decline, privacy concerns will be
diminished and limitations will be minimal.
Interviews will take place in Hutchings and Sis offices in order to provide privacy
behind closed doors for participants to feel safe and supported. Moreover, familiarity with the
SDMA office suite may also contribute to participant comfort. Chronologically, interviews will
occur based on Hutchings and Sis schedules, but primarily from 12:00 pm to 5:00 pm between
Monday and Friday to provide flexibility in scheduling around participant class and work
schedules and accommodating for students that sleep late. As suggested by Adams (2010),
interviews will be scheduled for 60 minutes in length. Given the relatively small participant pool
and hopeful community that would have been built by the program, no incentives will be
provided to participants other than the opportunity to advance the program and improve the
experiences of future mentees.
Data Analysis
The interviews will be analyzed through the transcripts and interviewers notes. Our
semi-structured interviews will be recorded and transcribed. Since this is the first time our study
is taking place, we will utilize an inductive approach by analyzing specific data and categorizing
them into themes and generalities. We will use descriptive coding to manage and organize data
as we sift through it (Rogers & Goodrick, 2010). Additionally, pattern coding will be utilized to
identify themes across interviews. Moreover, attribute coding, through the use of the
demographic form, will be used to compare demographic codes across particular groups of
students. Using all three coding techniques will answer both processes and outcomes questions
by identifying major trends in both assessment approaches and acknowledging how the program
may be affecting students with differing identities. More specifically, through the pawing
technique, we will examine each transcript in its entirety while pulling out quotes we feel best
represent the participants story of their experience in Brothers for Excellence. We will then
individually assign themes to our interview quotes before coming together to discuss our
findings.
To ensure the validity of our data analysis, we will be utilizing member-checking. For
this particular study, we will have six raters built from the four full-time professionals and two
graduate assistants in SDMA. For a theme to be coded into the results, we will require all six
raters to agree on the theme. While they do not have to fully agree with the theme itself, all
members must agree to have the theme present in the results, thus incorporating inter-rater
reliability. While this may make the process more difficult, it will increase the trustworthiness
and validity for the study. Of course, each rater will bring their own unique perspectives,
experiences, and biases into the coding and rating process. To account for this, each rater will
write out a reflective positionality statement. Each statement will then be shared with the group
and discussed.
Triangulation will also be utilized as another measure to ensure validity in the study. We
will use observations from staff that work with Brothers for Excellence (i.e., Hutchings and Sis),
relevant literature on men of color in the college environment and student engagement, the
results from the survey, and data mined from the in-depth interviews. Our goal is to avoid
making incorrect assumptions on how to positively change the program.
While we are taking many steps to ensure the most valid results possible, there are
limitations to our methods. Firstly, there will be a wide array of data to sift through. This will
put a strain on resources, especially time and energy, and allow for raters to potentially miss
important information. Working efficiently to code the data swiftly will help take some strain
off of the SDMA staff, while using the six raters will be our best way to combat missing key
information. Another limitation is we will not be interviewing all Brothers for Excellence
participants, which may narrow the data gathered, reducing the number of perspectives
represented. However, given the number of other priorities Hutchings and his staff maintain, this
is a necessity and will suffice.
Presentation of Results
In the final assessment report, results will be presented in both table and narrative
form. Firstly, relevant results will be categorized by their descriptive codes as the headers of
tables. The codes selected will be those most relevant to the outcomes and processes being
evaluated. Tables will then be filled with direct quotations that best exemplify those
codes. Quotations will be utilized to provide quick highlights of relevant themes, and tables will
serve as an organized method of categorizing those insights. Furthermore, a short narrative of
the findings will accompany the tables to briefly summarize and make meaning of the results, as
well as clarify quantitative findings. The tables and narrative will support one another in
presenting a cohesive account of findings.
Limitations
Within this assessment plan, there are a few limitations that must be addressed to obtain
the most valid and reliable results. We hope these limitations could be limited in future program
evaluations.
The first limitation is the swath of items that could and should be assessed with the
Brothers for Excellence program. While this assessment plan does its best to assess both
outcomes and processes, it cannot touch upon everything. For example, each large group
program could be assessed for both outcomes and process, which may provide quicker insights
than the cumulative assessment proposed in this plan. Yet, time, energy, and other
considerations direct evaluation in other areas. Although other assessment questions should be
explored in the future to garner as much data as possible, this plan focused on only a few areas,
thereby possibly excluding data and subsequent analyses that could be useful for student
development and more informed student affairs practice.
Moreover, the length and depth of the quantitative survey serves as a limitation. With
over 80 questions, participants may experience fatigue or apathy due to the length and reflection
required to answer accurately. Additionally, the timing of the survey at the end of the academic
year may not be conducive for memory recall and may limit results.
Within the qualitative section, the choice of interviewers may influence participant
responses. As mentioned, Hutchings and Sis were determined to be the most appropriate
interviewers due to their extensive knowledge of the participants and program. While
advantageous, participant familiarity with and affinity for these individuals may cause
participants to alter their answers to avoid conflict with the interviewers or provide expected
answers.
Timeline
The Brothers for Excellence assessment timeline (see Appendix I) was constructed to
allow time for flexibility but will still call for efficient work. Within the first two months, the
assessment plan will be created (or in this instance, edited). In April, the assessment plan will be
shared with key stakeholders and given to Campus Labs for distribution. The next step will be
alerting participants the quantitative survey will be administered at the final Brothers for
Excellence meeting in early May, with a few days scheduled to reach out to any participants that
may have missed the meeting. The communication of this will be very important in efforts to
reach our goal of at least a 90% response rate. Once the survey has been administered, SDMA
will analyze the quantitative data during the summer months, an appropriate time for analysis
due to the natural down time of the summer break. In the month of July, SDMA will reach out to
participants who indicated they would like to participate in the semi-structured interviews,
confirm interview participants, and schedule interviews for August and September. Finally,
September, along with the rest of the fall semester, will be used to code, analyze, and determine
significant findings.
Budget
No budget is necessary or will be provided for this program evaluation. All assessment
will be conducted by SDMA staff, all of whom have experience with program evaluation as
either masters level professionals or students in or having taken the appropriate classes.
Additionally, no incentives will be provided to participants, as surveys will be required at a
program meeting and more likely than not enough participants will be willing to interview.
However, given the other responsibilities of the SDMA staff, it may be beneficial for future
evaluations to be done by alternate parties for the sake of time and energy, though this would
require a budget if done so.

Next Steps
Brothers for Excellence is currently in its fifth year of existence and has yet to be
evaluated officially, and as such, it is vital to the program that both outcomes and processes are
evaluated in order to truly assess if the students are receiving the desired goals of the program.
Regarding next steps, evaluators must first determine significant implications for practice from
both quantitative and qualitative measures. Secondly, evaluators must also reflect upon and
apply those implications to the current program structure, determining which components of the
program to eliminate, alter, or add. However, those decisions must be made within institutional,
departmental, and environmental contexts to consider external factors that may affect the
program. Finally, Hutchings will need to reorganize the program appropriately for re-launch,
which may be mid-year adjustments or for fall 2018.
This is an important time for Brother for Excellence. Hutchings will be making
important decisions that will affect the program, and more importantly, its participants for the
foreseeable future. The qualitative and quantitative approaches laid out in this assessment are
vital to the overall advancement of the program, and we hope that this evaluation plan will
improve the experiences of men of color at Loyola and prepare them for success in their
personal, academic, and professional endeavors.

References
Adams, W. C. (2010). Conducing semi-structured interviews. In J. S. Wholey, H. P. Hatry, & K.
E. Newcomer (Eds.), Handbook of practical program evaluation (3rd ed.) (pp. 365-377).
San Francisco, CA: Jossey-Bass.
Banta, T. W., & Palomba, C. A. (2015). Assessment essentials: Planning, implementing, and
improving assessment in higher education (2nd ed.). San Francisco, CA: Jossey-Bass.
Dancy II, T. E. (2011). Colleges in the making of manhood and masculinity: Gendered
perspectives on African American males. Gender and Education 23(4), 477-495.
Keeling, R. P., Wall, A. F., Underhile, R., & Gungy, G. J. (2008). Assessment reconsidered:
Institutional effectiveness for student success. Washington, DC: NASPA.
Lester, J., & Harris III, F. (2015). Engaging undergraduate women and men. In S. J. Quaye
& S. R. Harper (Eds.), Student engagement in higher education (2nd ed.) (pp. 149-170).
New York, NY: Routledge.
Quaye, S. J., Griffin, K. A., & Museus, S. D. (2015). Engaging students of color. In S. J. Quaye
& S. R. Harper (Eds.), Student engagement in higher education (2nd ed.) (pp. 15-25).
New York, NY: Routledge.
Rogers, P. J., & Goodrick, D. (2010). Qualitative data analysis. In J. S. Wholey, H. P. Hatry, &
K. E. Newcomer (Eds.), Handbook of practical program evaluation (3rd ed.) (pp. 429-
453). San Francisco, CA: Jossey-Bass.
Strayhorn, T. L. (2008). The role of supportive relationships in facilitating African American
males success in college. Journal of Student Affairs Research and Practice 45(1), 26-48.
Strayhorn, T. L. (2011). When race and gender collide: Social and cultural capitals influence on
the academic achievement of African American and Latino males. The Review of Higher
Education 33(3), 307-332.
Student Diversity and Multicultural Affairs (2016). Brothers for excellence (b4e). Retrieved
from http://www.luc.edu/diversity/programs/moci/
W. K. Kellogg Foundation. (2004). Using logic models to bring together planning, evaluation,
and action: Logic model development guide. Battle Creek, MI: Author.
Appendix A
Brothers for Excellence Contract
Brothers for Excellence Mentee Contract

Brother for Excellence Mission Statement:
The mission of the brothers for excellence program is to assist men of color achieve success in all aspects of life
and satisfaction as members of the Loyola community. In connection with the Jesuit tradition, the program seeks
to provide care for the entire person cura personalis as we guide, support, and challenge our men of color. To
empower and cultivate relationship with men of color to discover, connect, and transform at Loyola and beyond.
B4E Program is intended to fulfill the following:
Foster meaningful relationships with B4E brothers, peer mentors, faculty, and professional staff of color
Enrich and strengthen connection of men of color within the Loyola community
Increase retention rates among men of color and track academic progress of all participants
Facilitate leadership development among brothers
Cultivate personal empowerment among brothers
As a mentee of B4E, I am expected to:
Self-identify as a Man of Color

Attend B4E Welcome/Kickoff (Wednesday, August 31, 2016, IC 4th Floor, 5-7PM)
Attend B4E Mentor/Mentee Mixer (Wednesday, September 7, 2016, McCormick Lounge, 5-7PM)
Attend Monthly Community Engagements (Sept 28, Oct 26, Nov 30, Jan 25, Feb 22, Mar 29, Apr 12)
Social Outings (Sep 18 and Apr 15)
Attend B4E Retreat at LUREC (Friday to Sunday, October 14-16)
Attend B4E Mentee Summit (Saturday, November 19, Beane Hall WTC, 10-2PM)
Celebration of Brotherhood (Monday, April 24, 2017, IC 4th Floor, 5-7PM)
Set up monthly one on ones with assigned mentee(s)
Upon agreeing to participate in the Brothers for Excellence Program I agree to:
Make a one year commitment to being matched with my faculty/staff and peer mentor(s).
Meet my faculty/staff and peer mentor(s) at mutually agreed upon times at least once a month.
Be on time for scheduled meetings and events.
Notify B4E staff and mentor(s) at least 24 hours in advance if I will not be able to attend an event or
meeting.
Inform B4E staff if any difficulties or concerns arise throughout the duration of the mentor/mentee
relationship.
I understand that when my mentorship match ends, future contact with my mentor(s) is beyond the scope of the B4E
Program and can happen only by the mutual consensus of the mentor and mentee.
Print Name: ___________________________
Signature: ___________________________
Date: ___________________________
Appendix B
Logic Model for Brothers for Excellence Evaluation

Appendix C
Quantitative Survey Instrument

Appendix D
Quantitative Survey Matrix

Appendix E
Email Templates
Participant Outreach Email

Dear ____________ (potential participant),
My name is ______________ and I am an evaluator for the Brothers for Excellence (B4E)
assessment project. You are receiving this email because you answered yes to participating in an
interview about your experience with this program when completing the B4E survey.
Participation in the interview is completely voluntary, so please do not feel pressured to
participate. The purpose of the interview is to gain a more in-depth understanding of your
experience in the B4E program. We then hope to use the information gathered from your
interview to improve the program for future participants. If you are still interested in
participating, please email me back by ____________(date). We look forward to talking with
you and learning about your experience. Thank you and have a great day!
Best,
__________________(email signature)
Participant Acceptance Email

Dear ___________(participants name)
Thank you very much for agreeing to participate in our interview to gather more information
about you and your experience in Brothers for Excellence. This email serves as your official
confirmation for your interview, which will take place on _________(date) at _______(time).
The interview will take place in __________(location) and will last one hour. If you will need
any accommodations to complete the interview, please let me know. As a reminder, you will
have to fill out a consent form before the interview begins. If you have any questions, please feel
free to contact _____________(evaluator name) at ________(email and/or phone #). We look
forward to your interview!
Best,
_______________(email signature)
Appendix F
Informed Consent Form
CONSENT TO PARTICIPATE IN INTERVIEW FOR BROTHERS FOR EXCELLENCE

Project Title: Brothers for Excellence Evaluation
Interview Introduction and Purpose:

You are being asked to participate in an interview to help evaluate the effectiveness of the
Brothers for Excellence program. We hope to use the results of the interview to gain a more in-
depth understanding how Brothers for Excellence affects participants and to understand your
experiences within the program. You are being asked to participate in this interview because you
chose yes when asked if you were interested in participating in an interview at the end of the
B4E survey. Please take the time needed to read through this form carefully. Feel free to ask any
questions regarding the interview process before you decide to continue in the interview.
Procedures:
If you agree to continue with the interview, you will be asked to respond to a series of questions
for roughly one hour (60 minutes). We ask that you are open and honest when responding to
ensure we gain insight into your entire B4E experience. Please note that if you do not feel
comfortable answering certain questions, you are not required to answer.
Risks:
There are no known risks to participating in the interview and your responses will not affect your
involvement in future Brothers for Excellence involvement opportunities. Benefits to
participating in the interview will be a chance to improve the Brothers for Excellence experience
for future men of color participating in the program.
Confidentiality:
Your name will not be connected to any of the responses for this interview. We will analyze the
interview and if we have any questions about content, we will ask you for your feedback. The
information gathered in todays interview will only be shared with limited members of the
Student Diversity and Multicultural Affairs staff.
Contacts and Questions: If you have any questions about the interview you can email
qhutchings@luc.edu.
Statement of Consent: Your signature below indicates that you have read and understood all the
guidelines above, asked the necessary questions, and agreed to participate in the interview. You
will be given your own copy of this consent form as well.
Participant Signature: ______________________________ Date: ______________________
Evaluator Signature: ______________________ Date: ______________________
Appendix G
Demographic Variable Form

Appendix H
Interview Protocol
Preliminary
Greet interviewee, introduce yourself, give consent form, permission for recording etc.
Make sure to have reviewed any materials necessary to complete the interview in the
most effective manner. Make sure the interviewee is comfortable and has everything they
need to participate in the interview.
Explain the purpose of the interview and project as a whole.
o The purpose of this interview is to gain a better insight into your experiences
while participating in the program Brothers for Excellence. As you may
remember from taking the survey, we are looking to assess if the Brothers for
Excellence program is effective in creating a support group for participants and
helping you to be successful in your first year on campus. Formal assessment like
this has never been done on B4E, so we are excited to learn more about this
program and what it can do for students.
o The reason we are conducting interviews is to go deeper into the topics already
explored in the survey so as to more clearly hear your stories and experiences. We
would also like to use any and all information we collect through the survey and
participant interviews (such as this one) to improve B4E for future participants in
the program.
Explain how the interview will work
o I have some questions written down but really want to be guided by the answers
and insight that you give. I want to understand your specific experiences with
Brothers for Excellence and how they affected your first year at Loyola
University Chicago. So, if you feel like we are getting off topic or going on a
tangent, please do not worry. My overall main objective today is to LISTEN TO
YOUR STORY. Any information you deem important enough to tell me today is
important enough for our study.
o Please know I may be taking notes of some kind to better guide our interview and
better grasp your story. Please do not feel as if I am not listening when taking
notes, I promise I will be fully engaged during the entire interview.
Part I- Overall Experience with B4E

I want to start by getting an idea of your overall experience while in Brothers for Excellence. We
will then work down into more specific portions of the program. So to start
o How did you first hear about Brothers for Excellence?
o Probes: Can you tell me about what prompted you to begin participating in the
program? Why or why not did you want to participate?
o What were the first few weeks of being in Brothers for Excellence like?
o Probe: Did you have a group similar to B4E in high school?
o How has your relationship with B4E evolved over time?
o Probes: Can you discuss any benefits youve experienced as a result of Brothers
for Excellence? What about any obstacles youve experienced because of the
program?
o Could you please discuss your experiences participating in the large scale B4E programs
with all participants and mentors?
o Probes: Were there any programs that stood out to you? Why did you like certain
programs in particular?
o Would you be willing to participate in future B4E events as an alumnus of the program if
offered?
o Probes: What elements of your experience would lead you to do so or not?
So, from what you have shared your experience with Brothers for Excellence has been
_____________ (Provide 2-3 key ideas from what they have shared. Affirm that this is correct. If
not, seek clarification).
Part II- Academic Experience

Academics have the potential to impact your first year experience. So I want to ask you about
your academic experience during your time in B4E
o How have your grades been during your first year at Loyola?
o Probes: In what ways do you think B4E could address supporting its participants
in their academic performances?
o What academic resources did you use on campus?
o Probes: In what ways did you utilize those resources? How did you learn of those
resources?
o For what reasons are you planning on returning to Loyola in the fall? For what reasons
are you not planning on returning?
o Probes: In what ways could B4E
So, from what you have shared your academic experience while in Brothers for Excellence has
been _____________ (Provide 2-3 key ideas from what they have shared. Affirm that this is
correct. If not, seek clarification).
Part III- Mentor/Brother Connections

You interacted with a lot of different people during your time in Brothers for Excellence. This is
an intentional aspect of the program and I would love to hear about the connections you have
made during your time in B4E
o If overall survey results indicated mentor(s) helped with program outcomes:
o Overall, program participants felt that their peer mentors and/or faculty/staff
mentors helped them grow in terms of several of B4Es learning outcomes. Im
wondering whether that was the case for you and for what reasons your mentor
either did or did not do this?
Probes: In terms of identity development? Connection to campus? Sense
of belonging? Ability to succeed in academics?
o If overall survey results indicated mentor(s) did not help overall with program outcomes:
o Overall, program participants felt that their peer mentors and/or faculty/staff
mentors did not help them grow much in terms of several of B4Es learning
outcomes. Im wondering whether that was the case for you and for what reasons
your mentor either did or did not do this?
Probes: In terms of identity development? Connection to campus? Sense
of belonging? Ability to succeed in academics?
o What was your relationship like with the other men in Brothers for Excellence?
o Probes: Did the men in B4E become a support group for you?
So, from what you have shared your experience making connections within Brothers for
Excellence has been _____________ (Provide 2-3 key ideas from what they have shared. Affirm
that this is correct. If not, seek clarification).
Part IV- Identity Development

I want to ask next about whether B4E has helped you understand your social identities during
your first year at Loyola. To begin
o What was your understanding of your identities before coming to college?
o Probes: Which identities were most salient/important to you?
o How did Brothers for Excellence affect your development of social identities?
o Probes: How have you grown in terms of your male identity? Your person of
color identity? Do you have an understanding of the intersectionality of your
identities [and what can you attribute that to?]?
So, from what you have shared your identity development within Brothers for Excellence has
Part V- Campus Engagement

Lastly, Id like to hear more about your involvement on campus while a participant in Brothers
for Excellence
o Did either of your mentors encourage you to get involved in anything on campus?
o Probes: Did friends from the program? Did anyone in the program encourage you
to get more involved? Have you become a leader in anything as a result of B4E?
So, from what you have shared your campus involvement within Brothers for Excellence has
So with the last few minutes of the interview, I want to see if there is there anything else we did
not touch upon during the interview you feel is vital to understanding your experience in
Brothers for Excellence?
Thank you for sharing your thoughts today during this interview process. We appreciate your
time in helping us better understand the Brothers for Excellence experience. If you have any
additional comments you were not able to address today, please email Quortne Hutchings at
qhutchings1@luc.edu.
Appendix I
Evaluation Timeline
Appendix J
Final Powerpoint Presentation


Mageary and Randolph Elps431 Final Evaluation Plan

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Mageary and Randolph Elps431 Final Evaluation Plan

Enviado por

Direitos autorais:

Formatos disponíveis

Running head: BROTHERS FOR EXCELLENCE EVALUATION PLAN 1

Brothers for Excellence Evaluation Plan

Colin Mageary and Patrick Randolph

Loyola University Chicago

Brothers for Excellence Evaluation Plan

assistants. Quortne Hutchings, a Program Coordinator, oversees the program, including

in Woodstock, Illinois (Hutchings, personal communication, September 14, 2016). Politically,

Excellence specifically has an annual budget of $10,500, $4500 of which is designated

specifically for an annual retreat.

According to Quaye, Griffin, and Museus (2015), students of color at predominantly

many predominantly white institutions, leading to feelings of detachment, alienation, and

American society encourages men to conform to narrowly-constructed and stereotypical

when entering the college environment.

inclusive and ground their actions in the literature.

especially at predominantly white institutions.

Loyola University Chicago campus, including involvement and developing awareness of

Excellence strives to promote engagement, mentorship, and camaraderie among participants to

(personal communication, September 14, 2016).

extracted from the above synthesis.

accepted to Loyola University Chicago (Hutchings, personal communication, September 14,

In addition to one on one conversations, the department offers large-scale programming

than a one-year program (Hutchings, personal communication, September 14, 2016). A

participants do following the completion of the program, such as academic achievement,

leadership skills developed and opportunities to enhance their involvement on Loyolas

succeed to continue offering helpful opportunities and resources for marginalized

communication, September 14, 2016).

As those presenting this evaluation plan, we consider ourselves indirect stakeholders as

prosper as a result of our efforts.

divide their attention.

Additionally, while budgetary resources have not fluctuated, it could be a significant

the program with more credibility.

willingness to participate as alumni, and professional development?

and outcomes (see General Assessment Approach for more information).

by the Kellogg Foundation (2004), a logic model is a visual representation of a programs

they are at achieving their goals.

decides to alter the program in future years.

successful in fostering these learning outcomes?

General Assessment Approach

Potential strengths to this assessment approach are the commitment to program

individual mentee and may vary, which is significant to remember.

positions obtained. Secondly, we aspire to measure mentees identity development in terms of

Excellence program, namely first-year or transfer students that identify as men of

given the small population of approximately 40 to 45 mentees. Convenience and census

small population of approximately 45 students. As such, probability sampling strategies would

not be beneficial and limit data gathered.

Survey and Evaluation Design

receive survey responses from at least 90% of B4E participants.

Excellence and will be collected at the end of the academic year.

The quantitative survey instrument (see Appendices B and C) consists of 82 questions,

Programs, followed by a short section on participant demographics. Hutchings, as the Program

Coordinator for Mens Initiatives, will be listed as the primary contact.

include predetermined questions, but do not necessarily have to be implemented in a strict

share different information. Moreover, we elected for semi-structured interviews instead of

support quantitative methods by investigating survey data further following analyses of

participant recruitment are outlined in Appendix E. Additionally, Appendix F is an informed

class standing, conduct record, demographic, or experience, although participant demographic

Mentor/Brother Connections, Identity Development, and Campus Engagement, each of which

survey data and understand how processes inform outcomes.

would not benefit our evaluation of Brothers for Excellence.

more complete representation of participant experiences. Interviews will be conducted by

diminished and limitations will be minimal.

experiences of future mentees.

making incorrect assumptions on how to positively change the program.