Preparing for the On-Site Monitoring Review Community-Based Organizations

1|Pa g e

Table of Contents
Administrators Documentation Interview Form................................................3 Classroom/Tutoring Checklist..........................................................................14 Class Profile.......................................................................................................17 Tutoring Profile...................................................................................................18 Monitoring Notes for Classroom/Tutoring Observation..................................19 LEIS Monitoring Checklist.................................................................................20 Corrective Action Plan.......................................................................................22 Interim Follow-Up Report...................................................................................23 Five-Year Monitoring Schedule.........................................................................24 Sample Desk Monitoring Report.......................................................................26 Preparation Checklist........................................................................................28 Sample Schedule...............................................................................................30

2|Page

NORTH CAROLINA COMMUNITY COLLEGE SYSTEM Division of Academic and Student Services Basic Skills Monitoring Review For Use during Administrators Documentation Interview

College/Agency: Monitoring Team:

Contact Person: Date of Visit:

This document is designed for use during the administrator interview of the NC Basic Skills On-site Monitoring Review. It is based on the NC Program Standards that align with the 13 federal required program practices and the National Reporting System. To ensure quality performance, however, programs are encouraged to also assess their operations with the remaining program standards that are not included in the monitoring review. Each federal practice includes one or more of the following three items: 1. Actual data results related to the federal practice and NC Program Standards. Data results are indicated in bold type. 2. A list of documents the administrator should provide during the on-site interview. 3. A list of questions that will be addressed during the interview between the administrator and monitoring team member. Note: The System Office will provide the data for items in blue prior to the monitoring visit. You do not need to compile these data. Because some community-based organizations provide both classroom instruction and individual tutoring as well as paid staff and volunteers, the terms classroom and instructors are often used interchangeably with instructional setting and tutors.

Federal Practice 1: All students have measurable goals. NC Program Standards

3.4 Program has a clearly articulated goal setting and advising process which helps students design an appropriate social, academic, and career pathway.

Data/Document Review

Ye s

No

In Progress

1. Number and percentage of students

with follow-up goals (entered and retained employment, GED/AHS, entered postsecondary) Note: This number/% may be low depending on functioning level and goals of students. Enter employment Retain employment Obtain GED/AHS Enter postsecondary

Interview Notes
What type of training, technical assistance, and support are you providing to assist instructors/tutors in helping students set realistic personal and NRS goals?

3|Page

Federal Practice 2: The program can demonstrate its past effectiveness in improving the literacy skills of adults and families. Ye No In NC Program Standards Data/Document Review
s

Progress

2.16 An annual program evaluation process is conducted and includes mechanisms for receiving input from key stakeholders, such as staff, students, partnering agencies, and employers; data analysis; and review of relevant researchbased practices. 8.1 Program meets or exceeds state benchmark percentages for completion of each of the Educational Functioning Levels. 8.2 Program meets or exceeds state benchmark percentage for students obtaining employment. 8.3 Program meets or exceeds state benchmark percentage for students retaining employment. 8.4 Program meets or exceeds state benchmark percentage for students entering postsecondary education or job training. 8.5 Program meets or exceeds state benchmark percentage for students earning a high school credential or GED. Notes:

1.

Learning Gains: EFL completions

2. Follow-Up Goal Completions:

Employment, retention, postsecondary, GED/AHS 3. Administrator should have copies of: Annual program improvement plans, Evidence of staff meetings that address performance data analysis, and Description of program improvement strategies and results

Interview
Based on your performance results for level completions and follow-up goals, what do you feel are the greatest strengths of your program? What areas need improvement and why? Describe the programs annual evaluation process. How do you engage the staff and/or tutors in the program improvement process? How are results used to improve the program?

4|Page

Federal Practice 3: The program is committed to serving individuals who are most in need of literacy services, including individuals who are low-income or have minimal literacy skills. Ye No In NC Program Standards Data/Document Review
1.8 Recruitment reflects a comprehensive approach that draws from multiple venues. A written recruitment plan includes a description of the target population, expected outcomes, activities describing personnel responsibilities and timelines, recruitment strategies, materials, and evaluation. 2.1 The program serves individuals who are most in need of literacy services, including individuals who are low-income or have minimal literacy skills. Number of ABE and ESL students served compared to the number of adult secondary students served (ABE/ESL students should be greater.) # ABE and ESL students # of Adult Secondary students Evidence that the beginning ABE and ESL literacy levels are being served Number of ABE Beginning Literacy and Beginning Basic students; number of ESL Literacy, Low Beginning, and High Beginning students; % of all students served Percentage of students leaving before attending 12 hours; percentage of students attending long enough to be post-tested; retention rate (75% - state benchmark % less than 12 hours = % attending long enough for post- testing = Retention Rate = Enrollment trends for past three years 2007 2008 = 2008 2009 = 2009 2010 = Administrator should have samples of program marketing materials available for review.
s

Progress

Interview
Describe your programs recruitment plan. Does your program set annual goals for recruitment? Does your program use recruitment activities other than printing schedules of classes or tutoring sessions? Do recruitment materials recruit students for all facets of your services? Do programs follow up on students who miss classes? What does your program do to keep students in class/tutoring sessions?

5|Page

Notes:

Federal Practice 4: The program is of sufficient intensity and duration for participants to achieve substantial learning gains. Ye No NC Program Standards Data/Document Review
s

In Progress

2.8 Program design provides adequate time and intensity for mastery of course content.

1. Average number of contact hours

meets or exceeds state average for CBOs State CBO Average: Program Average: 2. Administrator should provide: Class and/or tutoring schedules showing how often classes and/or tutoring sessions last and how they offer sufficient time for intensity and duration. Data on instructor or tutor/student ratio

Notes:

Federal Practice 5: The program uses instructional practices that research has proven to be effective in teaching adults. Federal Practice 6: Educational activities are built on a strong foundation of research and effective educational practice. NC Program Standards Interview (followed by classroom/tutoring observation reports to verify)

6|Page

5.5 Programs use a variety of instructional materials, including authentic reading materials (i.e. magazines, newspapers, work-related texts), learning games, Madison Heights/Lifelines, TV411, Crossroads Caf, Internet-based resources, manipulatives, and special materials for persons with learning and/or physical disabilities. 5.6 A variety of teaching methodologies (e.g., small group/large group, collaborative learning, themebased instruction, computerassisted learning, field trips, guest speakers, peer tutoring, role playing) is used to address various learning styles and needs. 5.11 Instructional staff members develop instructional plans that support research-based practices and reflect adult learning theory. Notes:

Does your program use Madison Heights/Lifelines, TV411, Crossroads Caf, On Common Ground, and/or English for All? What is the process in your program for ensuring that instructors/tutors are using a variety of adultappropriate and up-to-date materials? How are new instructional materials selected for purchase? Are your instructors/tutors using the NC Basic Skills ABE content standards for math, reading, writing, and/or technology? If so, how are they implementing them? How does your program ensure that instructors/tutors are using varied instructional methodologies such as individualized and small group instruction, peer tutoring, project-based instruction, etc.? If you offer classes, how are they structured, e.g., open entry, managed entry, combination? As the instructional leader, what are you doing to ensure the use of research-based and effective educational practices in your program? How do you make sure that staff is kept up-to-date on the latest adult education research? Can you provide some examples of what we will see in the classrooms or tutoring sessions related to this?

Federal Practice 7: Programs use computers and other advances in technology, as appropriate. NC Program Standards
5.12 Technology is used to enhance instruction and deliver instructional services at a distance.

Interview (followed by classroom observation reports to verify)

Are computers available for all students regardless of where class/tutoring occurs? If computers are not at every site, how are computers made available to all students? How are computers and other technology being used within your program, e.g., instructional tool only, direct computer literacy instruction, distance learning, etc.?

Notes:

7|Page

Federal Practice 8: The program gives students the opportunity to learn in real life contexts to ensure that an individual has the skills needed to compete in the workplace and exercise the rights and responsibilities of citizenship. NC Program Standards Interview (followed by classroom observation reports to verify) 5.1 A process is in place to ensure What is currently being done to give students the
that curriculum and instruction are contextualized to real-life applications including high demand jobs and aligned to relevant entry and exit assessment instruments. 5.3 The curricula integrate work readiness skills, such as problem solving, financial literacy, critical thinking, and self management. Notes: opportunity to learn in real life contexts? How is instruction being contextualized to the workplace? How is instruction being contextualized to specific occupational training programs or career interests of the students? If you serve Adult Secondary Education (ASE) level students, does your program have a college transition/preparatory curriculum to prepare students for college? If applicable, is your program actively involved in preparing students to earn Career Readiness Certificates?

8|Pa g e

Federal Practice 9: The program is staffed by well-trained instructors, administrators, and counselors. NC Program Standards
2.11Program assesses staffing needs and plans an appropriate staffing structure to address those needs. 2.13Program employs an intensive hiring process to ensure that potential staff have been thoroughly screened and judged based on the skills and knowledge needed to perform the Basic Skills job functions. 2.14Staff members meet certification requirements (if applicable) as required by the institution/program. 6.8 Staff members continuously improve their practices through regular participation in professional development options such as workshops, conferences, self-directed research, literature reviews, study circles, mentoring, classroom observations, etc.

Data/Document Review
1. Administrator should have the following documents or evidence available for review: List of full-time and part-time staff and tutors Evidence of degrees of staff, if appropriate Training schedule for any locally or regionally offered PD Documentation of training attended by staff and tutors Percent of funding designated for training

Ye s

No

In Progress

Interview

Notes:

Have staff members or tutors attended the following training activities sponsored by NCCCS: o regional and statewide training workshops o Appalachian State University workshops/institutes o New CBO Directors Orientation o CBO Directors Institute? What other training activities have staff members and tutors attended? What type of in-house training is offered for program staff and tutors? Are new staff members and tutors required to have preservice training? Do your new instructors/tutors complete the Orientation to Basic Skills in NC online course? Are paid staff members required to attend training sessions annually? How many hours? Are volunteer staff members required to attend training annually? Have staff members been trained on the state assessment policy? Are tutors aware of the policies related to pre- and post-testing? Has at least one staff member been trained on LEIS and LACES? Have all staff members who give standardized assessments been trained in giving assessments? Can program describe changes and improvements that were made as a result of staff development activities?

9|Page

10 | P a g e

Federal Practice 10: The program coordinates its activities with other available resources in the community, including postsecondary schools, elementary and secondary schools, One-stop centers, job training programs, and social service agencies. NC Program Standards Data/Document Review
Ye s No In Progress

1.1 In collaboration with multiple

stakeholders, an assessment of needs related to moving adults with low basic skills to postsecondary education and/or employment is conducted, including, at a minimum: o the demographics of the target population in need of adult learning services; o the identification of high growth job clusters; o the educational and training providers located within the community; o available support services to foster retention, progress, and student transition; o available resources (e.g., funding, facilities, etc.) that can be used, shared, or coordinated; and o professional development and technical assistance needed for adult learning providers. Based on the results of the needs assessment, memoranda of understanding or similar agreements are developed among critical partners such as: o JobLink Career Centers or other organizations with workforce development capacity; o Occupational and postsecondary education providers, including Developmental Education, Continuing Education, and Curriculum; o Support service providers (e.g.,

1. Administrator should have:

a copy of the last full local plan/application that includes a list of partnerships. Copies of any written agreements with partnering agencies.

Interview
Are there any new partners since the plan was written? How and with whom do you partner to help students transition successfully to the workplace and/or postsecondary education, if applicable? Are there any new and/or exciting activities going on with partners?

1.2

11 | P a g e

o Notes:

transportation, child care, financial aid, counseling, career advising); and Business and industry.

Federal Practice 11: The program offers flexible schedules and support services (such as child care and transportation) that are necessary to enable individuals, including individuals with disabilities or other special needs, to attend and complete programs. Ye No In NC Program Standards Data/Document Review
s

Progress

2.2

A comprehensive orientation process for new students is conducted that minimally includes a welcoming activity, activities that focus on students strengths, preliminary goal setting, program policies and procedures, learning style inventories, career interest inventories, identification and resolution of potential barriers to participation, and

1. Administrator should have the following documents or evidence available for review: Class or tutoring schedule showing locations, days, and times of classes/tutoring sessions (day/evening, weekday/weekend) List of support services that are made available to students, including students with special needs

Interview

12 | P a g e

registration. Orientation should be followed by student assessment and formalized NRS goal setting. 2.6 Program ensures a safe, comfortable, and clean facility appropriate for adult students that meets the Americans with Disabilities Acts requirement of reasonable accommodations. 2.7 Classes/services are scheduled in locations that reflect community need. 3.3 The orientation process includes an educational counseling component which integrates a process for identifying potential barriers and support service needs to help students persist in the continuum of programs, making successful transitions along the way. 3.10 The program collaborates with partnering agencies to design a coordinated support services plan which provides community, educational, and/or workplace support services necessary to promote student retention and goal completion. Notes:

How do you assist students with transportation, child care, or other barriers to participation? Do you have a policy for serving adults with special needs? What types of accommodations have you provided in the past? What options do you have for keeping students connected to the program if they must exit temporarily, e.g., distance learning, mail-in study packets, etc.? Do all students go through orientation? Is it group or individualized? What does the orientation include? Are career counseling services available for students if they need it? Are students with the goal of getting a job referred to any agency for job placement services? What is the programs plan for helping students transition to further education?

13 | P a g e

Federal Practice 12: The program has a high-quality information management system that has the capacity to report participant outcomes and to monitor program performance. Ye No In NC Program Standards Data/Document Review
s

Progress

7.3 Appropriate staff members receive training on data collection procedures, data entry, and procedures for producing and interpreting LEIS reports. 7.4 Program uses the statedeveloped LEIS form for collecting participant information as defined by the National Reporting System (NRS) and state requirements. 7.5 Student data are entered in Colleague/LACES in compliance with NCCCS reporting timelines and procedures. 7.6 Program updates records on an ongoing basis to provide real-time data for analysis and program improvement. 7.8 Program administrators run LEIS reports on a regular basis (at least quarterly) to check for errors, missing data, outof-range values and anomalous data. Program has documented procedures for correcting errors and resolving missing data.

1. Administrator should provide copies

of local program and class/tutoring reports that are generated and used to monitor program performance, enrollment, and attendance.

Interview
Describe your data collection and entry procedures. For example, a. When, how, and to whom do instructors/tutors submit student data from post-tests, achievements, goals, etc.? b. How often are data entered into LACES (e.g., daily, weekly, monthly, etc.)? c. How often are reports run to check for errors, missing data, out-of-range values, and anomalous data? d. What are your procedures for correcting errors and resolving missing data? e. Has the program implemented procedures to ensure accurate data? What training have your data entry staff members had on LEIS, LACES and procedures for producing and interpreting reports? How often do you provide data reports to instructors/tutors and other staff members to identify possible problems and needs for improvement?

Notes:

14 | P a g e

Federal Practice 13: The community has a demonstrated need for additional English as a Second
Language (ESL) programs.

NC Program Standards
1.7 A process is in place for continually assessing and addressing future needs of adult learners and the community, including ESL students, with regular input from employers and partnering agencies.

Data/Document Review
1. Administrator should provide the following documents or evidence, if applicable: Copy of waiting list for ESL students Evidence of ESL classes offered throughout the community where need is present Statistics/Census data on ESL population in programs service area

Ye s

No

In Progress

Interview
If you have a waiting list for ESL students, how long have you had one? How have you addressed this situation in the past? Are there other providers (e.g. community colleges) in the community that provide ESL instruction as well?

Notes:

National Reporting System: The program follows all guidelines of the National Reporting System. NC Program Standards
4.1 Program uses NCCCSapproved standardized assessments to measure the educational functioning level, progress, and gains of students. 4.2 Program ensures staff members administering assessments meet test publisher and NCCCS training requirements. 4.3 Program ensures staff members understand and have a copy of the state assessment policy. 4.4 All students attending 12 hours or more have on file a pretest score from a NCCCS-

Data/Document Review
1. The administrator should have the following documents or evidence available for review: Copy of assessment policy that is distributed to staff, including list of standardized assessments being used by the program Attendance records Professional development records of any staff members who administer standardized tests to document training, if applicable 2. Post-testing percentage State average: Programs:

Ye s

No

In Progress

Interview

15 | P a g e

approved standardized test. 4.5 Program ensures that all standardized assessments are administered in a proctored environment (timed, quiet, and monitored). 4.7 Program ensures staff members administer postassessments in compliance with test publishers guidelines and document all exceptions to recommended timeframes for post-testing. 7.7 Program follows distance learning provisions for counting contact hours as outlined in the states assessment policy.

How do you make sure that all program staff members understand the National Reporting System (NRS) including educational functioning levels, follow-up goals, assessment policy, etc.? Describe your pre/post testing process. For example: a. Does the instructor/tutor have access to all student assessment information? b. Have the test administrators been properly trained? c. Are appropriate test administration procedures being followed (timing of test, proper forms, use of locators, etc.)? d. What are your procedures for dealing with outof-range scores? e. Is the person who inputs data trained to catch possible errors? f. How does the program use its data to improve program quality? If you have students enrolled in distance learning, describe the process you use to track contact hours.

Notes:

16 | P a g e

NORTH CAROLINA COMMUNITY COLLEGE SYSTEM Division of Academic and Student Services Basic Skills Classroom/Tutoring Checklist

College/Agency: Date:

Instructor/Tutor: Monitoring Team Member:

To obtain a closer look at the instructional practices of the Basic Skills program, the monitoring process includes a classroom/tutoring observation and interview component. This component includes: 1. Up to a 20 minute observation of classroom/tutoring instruction 2. A review of 5 randomly selected student and/or administrative folders 3. A 10 minute interview with the instructor/tutor 4. A 10 minute interview with the students The following checklists outline the practices and procedures that the monitoring team will be observing. The items are aligned to the NC Program Standards and required program practices under the federal Adult Education and Family Literacy Act. If possible, the program should provide a staff member who can oversee the class while the monitoring team interviews the instructor. All instructors should review the checklist prior to the onsite monitoring visit. We realize that the observation will provide only a snapshot of the instructional practices, but instructors/tutors are encouraged to incorporate a lesson that is representative of their daily practice.

1.

Classroom/Tutoring Observation
Observation Is the classroom/tutoring area conducive to learning (e.g., adequate heating, cooling, lighting; sufficient number of chairs and tables; clear signage to direct students to location; free of distracting noise; etc.) Is the classroom/tutoring site handicapped accessible? Are standardized assessments stored in a locked cabinet or other secure location? Do you see evidence of instructional activities that are contextualized to real life application? Is the instructor/tutor using a variety of adult-appropriate and up-to-date
17 | P a g e
Y N e o s * Not Observ ed

NC Program Standards 2.5 Program ensures a safe, comfortable, and clean facility appropriate for adult students that meets the Americans with Disabilities Acts requirement of reasonable accommodations. 4.6 Program stores standardized assessment materials in a secure environment. 5.1 A process is in place to ensure that curriculum and instruction are

contextualized to real-life applications. 5.5 Programs use a variety of instructional materials. NC Program Standards 5.6 A variety of teaching methodologies is used to address various learning styles and needs. 5.8 Classroom instruction models and reinforces real world expectations that enhance readiness for postsecondary education, training, and employment. 5.12 Technology is used to enhance classroom instruction and deliver instructional services at a distance.

materials? Is the instructor/tutor using a variety of teaching strategies and materials such as computer-assisted instruction, collaborative learning, peer tutoring, field trips, manipulatives, etc.? Observation If applicable, do you see evidence of instructional activities that support readiness for postsecondary education or employment, such as study skills, note taking, attendance policies, timeliness, team building, critical thinking, problem solving, career days, etc.? Are computers available to all students? Is there evidence that the instructor/tutor is making appropriate use of computers and/or other technology within the instructional setting?

Y N e o s

* Not Observ ed

*Any items marked not observed should be addressed in the instructor interview to determine the status.

2. Student Folder Review

NC Program Standards Folder Review
Five randomly selected student and/or administrative files are reviewed and include evidence of:
Y N e o s * Not Observ ed

3.4 Program has a clearly articulated goal setting and advising process which helps students design an appropriate social, academic, and career pathway. 4.1 Program uses NCCCSapproved standardized assessments to measure the educational functioning level, progress, and gains of students.
4.4 All students attending 12 hours or more have on file a pretest score from a NCCCS-approved

Measurable and realistic short term and long term goals based on real life needs and interests Appropriate selection of NRS followup goals on LEIS form (e.g., high school/GED, employment, postsecondary) based on functioning level and related factors Evidence that goals are reviewed on a regular basis and updated as needed Learning plan or student work related to student goals The use of state-approved assessment instruments for pre and post-testing (e.g., CASAS, TABE, BEST Plus/BEST Literacy, WorkKeys) Use of the most recent LEIS form to collect student data
18 | P a g e

standardized test. Pre-test type and form (e.g., TABE 9), 4.7 Program ensures staff score, date, and entry EFL members administer post Post-test type and form (different but assessments in compliance equivalent form), score, date, and with test publishers exit/progress EFL guidelines and document Documentation to show number of all exceptions to hours between pre- and post-tests recommended timeframes Documentation to justify post-test for post-testing. time exceptions, if applicable 7.4 Program uses the statedeveloped LEIS form for collecting participant information as defined by the National Reporting System (NRS) and state requirements. *Any items marked not observed should be addressed in the instructor interview to determine the status.

3. Instructor Interview Interview Questions Any items from folder review and classroom observation marked not observed will be addressed during the instructor/tutor interview. Describe the goal setting process that is used to develop both personal and NRS-related goals. How is assessment information used to assist in realistic goal setting? Who helps students set goals? How are goals updated? How do you keep up-to-date with the latest adult education research and best practices? Provide two examples of how you are integrating research-based practices into your classroom or tutoring session. How do you give students the opportunity to learn in real life contexts? Describe your instructional planning process. How do you determine what to teach? Do you develop lesson plans? Do you use the NC content standards for ABE level instruction? If so, how? Do you use a particular curriculum? If so, what do you use? How do you adjust instruction to match student needs, interests, and reallife applications? Give two examples of how you have transferred something you learned through professional development into practice. What accommodations do you provide for students with special learning needs, such as learning disabilities, physical impairments, etc.? Describe the process you use to determine when a student is ready to be posttested. Describe the process you use for submitting post-test results and/or goal achievements for data entry. Do you know what percent of your students have been post-tested? Do you know what percent of your students have completed their EFL s so far
19 | P a g e

Interview Questions this year? If so, how do you use that information to improve instruction? 4. Student Interview
What was the main reason that you enrolled in this program? When you first enrolled, what were your experiences like (e.g., student orientation and intake, first few days in class/with tutor)? Is the class or tutoring session what you thought it would be? If yes, in what way? If no, what did you not expect? What do you like best about the program? What could we do to improve this program?

Sample Interview Questions

Basic Skills On-Site Monitoring Class Profile (for use if CBO conducts small group classes)

A class profile form should be completed PRIOR to the on-site for every class that the program selects for an observation visit during the monitoring review. Class profile forms should be available for distribution to the monitoring team upon arrival. (If the monitoring team will be observing tutoring sessions, complete the tutoring profile form on the next page instead of this class profile form.)

Instructor/Tutor Name: ___________________________________________________________ Class Location: _______________________________________________________________________ Functioning Levels Being Served (circle all that apply): ___ABE ___ASE ___ESL Class Type: ___ Learning Lab ___ABE Class ___GED Prep ___Adult High School ___Compensatory Education ___ESL Class ___College Transitions ___EL-Civics ___Family Literacy ___Prison ___Pathways to Employment ___Workplace Literacy ___Small Group (5 or fewer) ___Other: ___________________________________________________________ Class Structure: ___Open Entry ___Managed Enrollment ___ Other Length of cycle: ______ Describe: ____________________ Class Schedule: Circle the days the class meets: M Tu W Th F Sat Indicate the hours the class meets: __________________________________
20 | P a g e

How long have you been teaching this particular class? ____________________________________ Enrollment: Number of students currently on the class roster: _______ Number of students attending on a daily basis: ______ Questions:
1. Circle any of these that you use within your classroom:
Madison Heights/Lifelines English for All TV411 Crossroads Caf On Common Ground

2. Place an x by any of these PD opportunities in which you have participated within the past two years.
Regional training workshops Topic/s: Local workshops/trainings Topic/s: Online Orientation to Basic Skills in NC Other (list) Appalachian State University workshops/institutes Topic/s: NCCCS Correctional Education Training Other online courses Topic/s: Basic Skills and Family Literacy Conference NC Literacy Association Trainings/Conference National Conference Which one?

21 | P a g e

Basic Skills On-Site Monitoring Tutoring Profile (for use with student/tutor pairs)
A tutoring profile form should be completed PRIOR to the on-site for individual tutoring sessions that the program selects for an observation visit during the monitoring review. Tutoring profile forms should be available for distribution to the monitoring team upon arrival. (If the monitoring team will be observing class sessions, complete the class profile form on the previous page instead of this tutoring profile form.)

Tutor Name:

___________________________________________________________

Tutoring Site: _______________________________________________________________________ Functioning Level of Student: ___ABE Tutoring Schedule: Circle the days the tutor/student meet: M Indicate the hours the tutor/student meet: ___ASE ___ESL

Tu W Th F Sat __________________________________

How long have you been working with this student? ____________________________________ How long have you been working as a tutor? ___________________________________________ Questions:
1. Place an x by any of these PD opportunities in which you have participated within the past two years.
Regional training workshops Topic/s: Local workshops/trainings Topic/s: Online Orientation to Basic Skills in NC Other (list) Appalachian State University workshops/institutes Topic/s: NCCCS Correctional Education Training Other online courses Topic/s: Basic Skills and Family Literacy Conference NC Literacy Association Trainings/Conference National Conference Which one?

22 | P a g e

Monitoring Notes for Classroom/Tutoring Observations

Class: 1. Classroom/Tutoring Observations Commens:
(for State use only)

Instructor: Commendations:

Recommendations:

2. Folder Review

Comments:

Recommendations:

Commendations:

3. Instructor Interview

Comments:

Recommendations:

Commendations:

4. Student Interviews

Comments:

Recommendations:

Commendations:

23 | P a g e

NORTH CAROLINA COMMUNITY COLLEGE SYSTEM Division of Academic and Student Services Basic Skills LEIS Monitoring Checklist

College/Agency: Date:

LEIS Staff Person: Monitoring Team Member:

This document is designed for use during the LEIS staff member interview of the NC Basic Skills On-site Monitoring Review. It is based on federal required program practices that have been aligned to various elements in North Carolinas Program Standards.

Federal Practice: The program has a high-quality information management system that has the capacity to report participant outcomes and to monitor program performance. Federal Practice: The program follows all guidelines of the National Reporting System. NC Standards *Interview Questions
4.1 Program uses NCCCSapproved standardized assessments to measure the educational functioning level, progress, and gains of students. 4.4 All students attending 12 hours or more have on file a pretest score from a NCCCS-approved standardized test. 4.7 Program ensures staff members administer postassessments in compliance with test publishers guidelines and document all exceptions to recommended timeframes for post-testing. 7.3 Appropriate staff members receive training on data collection procedures, data entry, and procedures for producing and interpreting LEIS reports. 7.4 Program uses the statedeveloped LEIS form for collecting participant information as defined by 1. What questions do you have for me?

Level Completions:
2. 3. 4. 5. 6. What processes are followed to place new/beginning students? What tests are used (CASAS, TABE, BESTP, etc)? Who determines when it is time to post-test? Who does the post-testing? How do you document level completions? How do you document level completions when no post-tests were given?

Goal Completions:
7. How do you document and record goal completions? 8. Do you have a method for checking student outcomes against their goals? What is done if a student achieves an outcome that was not a goal? Example: Passed GED but GED was not a goal. 9. Do you review a student's employment status when setting a goal of Get a Job or Retain a Job? 10.What kind of follow-up is done on separated students? 11.How are goal achievements reported and recorded on separated students? 12.How are contact hours for distance learners being reported and recorded?

Recording and Reporting

24 | P a g e

7.5

7.6

7.7

7.8

the National Reporting System (NRS) and state requirements. Student data are entered in Colleague/LACES in compliance with NCCCS reporting timelines and procedures. Program updates records on an ongoing basis to provide real-time data for analysis and program improvement. Program follows distance learning provisions for counting contact hours as outlined in the states assessment policy. Program administrators run LEIS reports on a regular basis (at least quarterly) to check for errors, missing data, out-of-range values and anomalous data. Program has documented procedures for correcting errors and resolving missing data.

13.Do you have training for any individuals in charge of completing LEIS paper work? 14.What is the process for recording instructor/tutor info? 15.What is the process for recording students contact hours? 16.When do instructors/tutors turn in outcomes and test results? Do you receive paper work in a timely manner? 17.What types of status reports do your instructors/tutors receive during the year? Do instructors/tutors receive statistics on their students outcomes? 18.Do you have access to LACES reports? 19.Do you use any of the LACES reports for monitoring data and student progress? Which reports do you use most/like best?

Technical Assistance
20.Do you have at least one person whose main responsibility is to record LACES data? 21.What do you need to help you do your job to best serve the students?

*Please note that LACES staff member will also be asked to demonstrate some of the data procedures on the computer during the interview process.

25 | P a g e

Corrective Action Plan

Program: Required Action 1:

Target Date for Completi on

These are the strategies/activities we will implement to address the required action.

This is how we will evaluate the success of the strategies.

Staff Assigned

Status (pending or complete d)

26 | P a g e

Basic Skills Corrective Action Plan Interim Follow-Up Report

Due Date: four months after submission of original Corrective Action Plan
Note: Outcomes and results should be directly related to the required action being addressed. They should be specific and measurable in terms of changes in student performance, instructional practices, policies or procedures, curriculum and instruction, etc. Please attach any documentation that you have, such as performance data, classroom observation forms, training agendas and participation, revised policies or procedures, etc.

Required Action 1: Strategies from original corrective action plan Status

(pending or date completed)

Measurable Outcomes and Results of Strategies

Required Action 2: Strategies from original corrective action plan Status

(pending or date completed)

Measurable Outcomes and Results of Strategies

27 | P a g e

BASIC SKILLS PROGRAM MONITORING Cumulative Schedule for Community Colleges and CBOs 2010 - 2011 through 2014 - 2015
Program Alamance CC Asheville-Buncombe TCC Literacy Council of Buncombe County Beaufort County CC Bladen CC Blue Ridge CC Blue Ridge Literacy Council Brunswick CC Brunswick County Literacy Council Caldwell CC&TI Cape Fear CC Cape Fear Literacy Council Carteret CC Carteret County Literacy Council Catawba Valley CC Central Carolina CC Harnett County Literacy Council Hoke Reading/Literacy Council Central Piedmont CC Cleveland CC Coastal Carolina CC College of the Albemarle Craven CC Craven Literacy Council Davidson County CC Durham TCC Durham County Literacy Council Orange County Literacy Council Edgecombe CC Fayetteville TCC Fayetteville Urban Ministries & Adult Reading Program Forsyth TCC YMCA Literacy Initiative Gaston College Gaston Literacy Council Guilford TCC Reading Connections Halifax CC Haywood CC Isothermal CC James Sprunt CC Johnston CC 20102011 20112012 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 28 | P a g e 20122013 20132014 X 20142015

Lenoir CC Martin CC Program Mayland CC McDowell TCC Mitchell CC Montgomery CC Nash CC Pamlico CC Piedmont CC Pitt CC Literacy Volunteers of America - Pitt County Randolph CC Richmond CC Scotland County Literacy Council Roanoke-Chowan CC Robeson CC Rockingham CC Rockingham County Literacy Project Rowan-Cabarrus CC Sampson CC Sandhills CC Moore County Literacy Council South Piedmont CC Literacy Council of Union County Southeastern CC Columbus County Literacy Council Southwestern CC Stanly CC Surry CC Tri-County CC Vance-Granville CC Wake TCC Literacy Council of Wake County MOTHEREAD Wayne CC Western Piedmont CC Burke County Literacy Council Wilkes CC Wilson CC Total

X 20102011 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 16 15 16 17 X 17 20112012 20122013 X 20132014 X 20142015

29 | P a g e

Sunny Community College

ARE THEY TRANSITIONING?
TRANSITIONI NG GOALS
# and % of Students with Goal # % of all with student goal s Target Percen t Percent Attaine d Met/ Unme t

Follow-Up Goals Performance Rating

Desk Monitoring Report 2008 2009

Placement in unsubsidized employment Retention in unsubsidized employment High school completion (GED & AHS) Placement in postsecondary education or training

63 104 307 178

2% 5% 10% 6%

20% 26% 50% 33%

49% 18% 75%

Target Average

Program Average

+ or -

32.25% 84%

56.5%

+2 4

* +1 required for compliance

*Transitioning Performance Rating

+2 4

The Basic Skills program supports instruction for adult learners who need to sharpen their skills to earn a high school credential, prepare to transition to postsecondary education or job training, prepare to seek employment, learn the English language, or increase involvement in their childrens education. This desk monitoring report examines how well we do that by providing insight into four critical questions about our students: Are they enrolling? Are they attending? Are they learning? Are they transitioning? Data are also provided on student demographics, retention, assessment, and funding.

ARE THEY ENROLLING?

COST PER STUDENT/COST PER HOUR

*A median was determined for four groupings based on the size of the programs state and federal allocation. Group 1: up to $500,000 Group 2: $500,001 - $1M Group 3: $1.1M - $2.5M Group 4: $2.6 +

ARE THEY ATTENDING?

30 | P a g e

DEMOGRAPHICS, RETENTION, ASSESSMENT

Educational Functioning Levels

ARE THEY LEARNING?

CORE INDICATORS OF PERFORMANCE
Programs Level Completion Target for Level Completion Average Contact Hours/Level UnmetMet/ Average Program Target Average

Level Compl Performance

ABE Beg. Literacy ABE Beg. Basic Educatio n ABE Inter. Low ABE Inter. High ASE Low

349 152

19% 38%

32% 55%

140 122 109

39% 38% 45%

54% 45% 54%

35.8%

48%

ESL Beg. 33 42% 33% Literacy ESL Beg. 84 45% 65% Low ESL Beg. 104 38% 53% High ESL Inter. 92 36% 46% Low ESL Inter. 85 34% 38% High ESL 116 15% 21% Advance d ASE High is not included because it is measured through high school/GED completion. * +1 required for compliance

35%

42.7 %

*Level Completion Performance Rating

Planning Checklist for Program Administrator

Before the Review Activity Confirm the date of the review with the System Office. Receive the notification letter and monitoring packet from the System Office. Notify your supervisor or board president of the date and request their attendance at the exit meeting.
I will complet e this on: Septemb er October Comple ted

31 | P a g e

Conduct staff meeting to discuss monitoring process with instructors/tutors, data staff, etc. (Sample powerpoint developed by System Office for your use) Distribute classroom/tutoring checklist to all instructors/tutors to review four components of classroom visits (observation, folder review, instructor/tutor interview, student interview). All instructors/tutors should review the form even if they are not being visited. Distribute LEIS interview form to LACES staff member for review. Select eight classroom/tutoring sites for visitation during the review. Sites should be representative of your programs services. Distribute class profile forms to instructors and tutoring profile forms to tutors at selected sites with directions for completing and returning them to you at least one week prior to the monitoring review. Remind them to review the classroom/tutoring checklist. Develop classroom/tutoring visitation schedule (Two teams will visit four classes/tutoring sessions each (total of eight) during one-hour intervals between 10:00 12:00 and 1:00 3:00.) See sample schedule for details. Submit detailed schedule to System Office (see attached sample schedule for guidance). Review the Administrators Interview Form, prepare responses to interview questions, and compile documents/data. If possible, assign individuals to accompany each of the two classroom/tutoring visitation teams so they can cover the classes/tutoring session while the monitors interview the instructors/tutors. Reserve a meeting room for monitoring team to use for initial, debriefing, and exit meetings. Touch base with LACES staff person and instructors/tutors being visited to identify any questions, concerns, etc. Select a location (restaurant) for lunch so the monitoring team and selected staff (if possible) can continue discussions while they eat. Collect class profile and tutoring profile forms from instructors/tutors and finalize monitoring documentation materials. Activity Confirm attendance of your supervisor/president at the exit meeting.
32 | P a g e I will complet e this on: Comple ted

During the Review Welcome and meet the monitoring team. Provide brief overview of your program, distribute class profile forms to monitoring team, and ask assigned individuals to escort monitors to class/tutoring sites. Ask local LACES staff member to escort LEIS monitor to office for data interview/meeting. Meet with lead monitor to review program documentation and complete administrators interview. Break for lunch around noon. Continue meeting with lead monitor until documentation review/interview is complete. Provide monitoring team a room for debriefing at 3:00 PM. Participate in exit meeting with supervisor, president, and monitoring team around 3:20 PM. After the Review Thank staff for their participation and assistance during the review. Receive and review the monitoring report (sent within three weeks from the System Office following the review). Share the monitoring report with staff, as appropriate. Invite input from selected staff in responding to any required actions. Submit the Corrective Action Plan (CAP). Review the post-CAP memo to determine if any questions or concerns from the System Office need to be addressed. If so, submit responses within two weeks of receiving letter. Begin implementing strategies outlined in your CAP making sure to track progress and outcomes. Four months after CAP was submitted, complete the Interim Follow-Up Form to report on outcomes of strategies completed to date. Submit subsequent follow-up forms, if needed.

33 | P a g e

Sample Monitoring Schedule

9:30 Monitoring team arrives. Welcome and introductions (Local staff are welcome to attend the opening meeting, if desired.) Brief overview of program by program director Distribution of schedule and class and tutoring profile forms for site visits Classroom/tutoring monitor #1 escorted to class/tutoring site. Classroom/tutoring monitor #2 escorted to class/tutoring site. LEIS monitor meets with local LACES staff member. Lead monitor meets with program director.

10:00

11:00 Classroom/tutoring monitor #1 escorted to another class/tutoring site. Classroom/tutoring monitor #2 escorted to another class/tutoring site. LEIS staff continue to meet. Lead monitor and director continue to meet. Noon be used for Lunch break at local restaurant (Note: Basic Skills funds cannot food.)

1:00 Classroom/tutoring monitor #1 escorted to another class/tutoring site. Classroom/tutoring monitor #2 escorted to another class/tutoring site. LEIS staff continue to meet. When finished, LEIS monitor will accompany class/tutoring visitations. Lead monitor and director continue to meet. 2:00 Classroom/tutoring monitor #1 escorted to another class/tutoring site. Classroom/tutoring monitor #2 escorted to another class/tutoring site. Lead monitor and director continue to meet. When finished, lead monitor will accompany class visitations. 3:00 3:20 4:00 Monitoring team meets for debriefing meeting. Exit meeting with program director; his/her supervisor or board president; and selected staff, if desired. Monitoring team exits. Celebrate!

34 | P a g e

35 | P a g e