Student Portfolios and Authentic Learning Outcomes Assessment A.

Rae Clementz Ron Pitt Faculty Development Center Bryant College November 17, 2002
Introduction There are two parallel initiatives at Bryant College currently under development. The first is that of a student professional portfolio a mechanism by which students can demonstrate their skills and achievements along a number of dimensions to potential employers and other external and internal constituencies such as advisors, colleagues, parents, scholarship committees, and graduate schools. The second is a form of authentic performance assessment a mechanism by which Bryant College can determine the degree to which the undergraduate program is meeting its mission and instructional objectives by examining the overall performance of its students. One common tool for conducting this kind of authentic performance assessment and that was strongly suggested in some of the Learning Outcomes Assessment Subcommittee reports in Spring 2002 is a portfolio. This commonality of terms has led to a series of legitimate questions about the purposes, forms, and functions of a somewhat nebulous Bryant Portfolio. The objectives of this paper are to:

Clarify the differences and areas of commonality between these two initiatives Propose a structure and process by which these two initiatives may be pursued while maintaining a degree of separation as well as connectedness

Process, Form, and Function of the Assessment Portfolio Although the professional portfolio and portfolio assessment share a word in their names, they represent different physical objects and are surrounded by different purposes, contexts, and procedures. Based on the assessment subcommittee reports, the assessment portfolio needs to have certain characteristics. 1. The skills and qualities associated with the undergraduate mission cut across numerous areas of the curriculum and are not discipline-specific. Rather than design or implement new assessment instruments, the assessment process should build upon existing coursework that is designated to be rich in particular learning outcomes (course-embedded assessment).

2. The complete group of stakeholders in a portfolio assessment must be identified, and their interests and concerns must be accommodated in a manner that ensures confidence in the results. In particular, student assessment must not, in any way, become bound up with or confused with faculty or course assessment.

3. The mechanisms by which the assessment portfolios are constructed must

be established in a manner that all the stakeholders are comfortable with the types of evidence, the confidentiality of the evidence, and the control over that evidence.

4. There must be either a complete or a representative sampling of the portfolios of the student population, in order for the full range of student learning outcomes to be apparent. 5. A clear set of acceptable types of artifacts or evidence of learning outcomes must be established, so that evaluating the artifacts does not become a case of comparing apples to oranges to bananas. 6. Associated with the range of acceptable evidence, quality benchmarks must be identified that determine the quality of student performance.

7. A process must be created by which the level of quality of the evidence can
be assessed with objectivity, validity, and repeatability. This may require the involvement of an external, disinterested third party.

8. Finally, a systematic review of the process must be developed and

implemented, to ensure that the process meets all needs. How Students View a Portfolio While the characteristics described above may serve the needs of a portfolio assessment at Bryant College, they do not necessarily meet the needs and objectives of the students. In general, the student concept of a portfolio tends to be that of a professional portfolio, i.e. a tool in gaining employment (Young, 2002). In an informal set of brief, voluntary interviews of students conducted over the lunch hours in the Rotunda during October 2002 by Rae Clementz, Bryant students were asked In what ways do you think a portfolio would be useful? All of the thirty-three students interviewed responded with some variation of to help me get a job, with one student also indicating that it might be helpful for getting into graduate school. When asked, What purpose should a portfolio serve?, again all students interviewed indicated that it should demonstrate their skills or achievements. Interestingly, roughly 30% of the respondents qualified their comments with of interest to the company Im interviewing with. This suggested that rather than an abstract and generic set of learning outcomes, students wanted a portfolio that demonstrated specific, job related skills. What was also implied, although never directly stated, was that students wanted a portfolio that was adaptable, so that different skills could be emphasized depending on the interests and focus of a potential employer.

The student portfolio can be further separated from an assessment portfolio in the types of evidence collected and the ways they are presented. An assessment portfolio is likely to contain whole projects or assignments and may even include associated interim deliverables. When asked what types of items should go into a portfolio, the majority of the students interviewed said samples of my best work. Several people even stated nothing from my freshman or sophomore classes. This difference was further reflected in their answers to the question What characteristics would a portfolio have? Twenty-nine of the thirty-three students indicated that the portfolio should look very professional. Somewhat surprisingly, when asked what format or delivery method would be most appropriate for a portfolio, the majority of students said it should be paper-based, although definitely electronically generated. When asked to explain why a paper-based format was best, the explanations revolved around not having access to computers during an interview, a lack of interest and/or lack of time on the part of the interviewer, and a desire to be consistent with perceived corporate practices. The format suggested by the students surveyed was consistent with the format corporate recruiters find most valuable streamlined, and with specific pieces of evidence of a skill, trait, or ability. Finally, there is the question of whether a portfolio should be an optional activity or a program requirement which would require a major restructuring of the curriculum at Bryant College. Both methods are being used at different schools with varying degrees of success. A member of the Educational Technology Development Group at the University of Washington was quoted as saying, If a student is putting a portfolio together just to satisfy some graduation requirement, he or she might feel resentful about the process, and the end result will probably not be very useful (Young, 2002). This sentiment was echoed at Bryant where slightly over half of the students interviewed indicated that they would create a portfolio, even if it were optional, and in some cases, especially if it were optional. One student commented that, making a portfolio is a way to differentiate myself from my peers. Im willing to go above and beyond whats required, and that makes me better. However, almost half the students said they would not create a portfolio unless required to do so. One of the students who was for making the portfolio a requirement explained, Id probably hate it while I was doing it, but when it was done, Id be glad I had done it, and it would probably help me a lot. Portfolios at Bryant College It is clear that, between a professional portfolio and an assessment portfolio, there are different needs with different objectives and constituencies that lead to different ends. However, it may be possible to leverage these two processes and objects to assist each other. The following table summarizes the differences and commonalities between implementing a professional portfolio and an authentic performance assessment.

Professional Portfolio Performance Assessment Primary Objective

Demonstrate skills desired by potential employers Assess student learning and performance along skills and qualities identified in undergraduate mission statement Students Faculty OPIR Reaccredidation committees Learning-Outcomes Assessment Committee Joint Committee on Advising and FirstYear Programs

Potential Stakeholders
Students Faculty Career Services Student Affairs Employers

Potential Audiences
Potential Employers Graduate Schools Scholarship Programs Faculty Advisors Parents, Friends, Associates Represents students best work Streamlined Adaptable to different audiences Professional in content as well as appearance Accessible to outsiders Faculty Department Chairs President and Vice Presidents Curriculum Committee Learning-Outcomes Assessment Committee OPIR Professional and Faculty Advisors Trustees Accreditation Bodies

Functional Differences
Represents actual student performance over time, for better or for worse Structured and assessed around the core objectives Built upon existing classroom activities Representative of all student work Accessible only to designated parties for assessment purposes

Proposed Next Steps and Timeline By December 15, 2002 Establish one committee of the major stakeholders, with two subcommittees, to define and answer the questions regarding the structure and process of these two initiatives By February 1, 2003 Begin a series of community forums on assessment and portfolios as a way to inform all stakeholders and initiate these projects By March 15, 2003 Develop a set of goals, objectives, and activities for each initiative in the coming semester and year By April 15, 2003 Establish an action plan for the implementation of these two initiatives Begin a series of community forums on the development of assessment criteria (rubrics) By June 15, 2003 Establish a pilot assessment rubric and identify a review body. Begin a pilot test of the portfolio plan in a single, voluntary summer session course. By September 1, 2003 Begin implementation of the portfolio initiatives Throughout Document these activities, and establish a procedure for assessing the progress of these two projects and recommending changes to the committees Governance of these two initiatives needs to maintain a degree of separation between them while building on the existence of each other. The attached chart is an attempt to identify the levels of reporting in the creation and functioning of the two committees. References Young, J. (2002, March 8). E-portfolios could give students a new sense of their accomplishments. Chronicle of Higher Education, 48(26), p. A31 Friedman Ben-David, M. (1999) AMEE Guide No. 14: Outcome-based education: Part 3 Assessment in outcome-based education. Medical Teacher, 21(1), pp. 23-25

Graphical representation of the inheritance structure in the creation and reporting of the two portfolio projects Administrative Units Presidents Office

Student Affairs Division

Academic Affairs Division

Faculty Administration

Information Systems Division

Governing Committees

Joint Committee on Advising and FirstYear Programs

Learning Outcomes Assessment Committee

Contract Committees

Operational Units

Student Senate Career Services

ACE Undergraduate Programs OPIR Faculty Development Center Portfolio Committees

IT Reaccreditation Committees

Professional Portfolio Committee

Performance Assessment Portfolio Committee