Evaluation Proposal

EDTECH 505-4172
A Proposal Submitted to Far West Laboratory for Educational Research and Development by Integrity Consulting

Christina Narayan | EDTECH 505-4172

EDTECH 505-4172 Christina Narayan 3/13/2012

Evaluation Proposal

Introduction The Far West Laboratory for Educational Research and Development (FWL) issued a request for proposal (RFP) for an evaluation of its Determining Instructional Purposes (DIP) training program. This proposal is from Integrity Consulting of Berkeley, California in response to the RFP of February, 2012. Description of Program Being Evaluated FWL developed the Determining Instructional Purposes (DIP) training program to provide training in educational administrative skills to plan effective school programs for school administrators and graduate students. The program consists of a training package to be used by a coordinator in a workshop or training course. The package includes a Coordinators Handbook and three training units. The design of the units allows the flexibility for one, a combination, or all the units to be used. Each unit provides step-by-step directions for a coordinator to follow. Each unit ranges from 155-259 pages and requires 10-18 hours. The program is inexpensive: Coordinators Handbook @ $4.50 and individual units @ 8.95 or all three for $24.95. Evaluation Method Primarily, Integrity Consulting will evaluate if there is a viable market for the DIP training program through a decision making model- Business Viability Model a. This model reviews five core dimensions: Market viability, technical viability, business model viability, management viability, economic and financial model viability and exit strategy viability. It is essential to review every aspect prior to investing money and time in the dissemination of the training units. Application of the Business Viability Model will provide stakeholders the essential commercial decision-making data. Secondly, this evaluation will provide valuable information for future school administrators deciding to purchase and utilize the DIP training materials. A common decision-making model will be implemented to evaluate this secondary area. The sources, instruments and other information are shown in Exhibit 2a. Information on the effectiveness of the training materials in addition to viability at the five core levels is necessary for this evaluation. The specific components of each dimension of the Business Viability Model are illustrated in Exhibit 1a. The stakeholders for this evaluation include FWLs project manager and other designated personnel, potential investors and potential clients. As result of this evaluation, FWL will first be able to decide in favor or against marketing and selling its DIP training program. If the report is favorably weighted in affirmation of the dissemination of its training units, strong assessment results from the evaluation will support marketing efforts.
Christina Narayan | EDTECH 505-4172

Exhibit 1a: Dimensions of Business Viability Model to Evaluate Primary Goal Dimension of Viability Market Viability Measure of Viability Measure Weighting Critical Weighting* Assessment** Validation*** Rec. Weight 25% Data Sources- All generated by Evaluation Team Marketing research report Rubric scored report for training materials Competition report

Technical Viability Business Model Viability Management Model Viability

Economic and Financial Viability Exit Strategy Viability

Market Environment: size, sustainability, potential market, target market, potential value Availability and quality of training materials, manufacturing process Competitive advantage, ability of competitor to duplicate and ability to create wealth Application of knowledge and skills, training, management of risk, and management of intellectual property Start-up costs, operating costs, raw material costs, overall profitability, ability to generate economic value Ability to define an exit strategy, ability to identify potential buyers, ability to schedule exit strategy

10%

15%

15%

Test results from Exhibit 2a

15%

Financial projection reports

20%

Exit strategy report

*Measure Weighting is the cumulative scoring out of 100 for the dimension as determined by Integrity Consulting **Weighting Assessment is the actual score assigned by Integrity Consulting ***Critical Validation is the overall scoring of each of the measures of viability for the dimension allotted by Integrity Consulting
Christina Narayan | EDTECH 505-4172

Evaluation Proposal

Exhibit 2a: Evaluation Design to Evaluate Secondary Goal Evaluation Questions Overall: Did the DIP training program have an impact on participants? 1.How do participants feel about their experiences after each unit? 2.Are participants more knowledge about setting goals after completing Unit 1? 3. Are participants better to analyze problems after completing Unit 2? 4. Are participants more accomplished at deriving objectives after completing Unit 3? Activities to Observe Data Source Data Collection Responsibility Data Analysis Audience

Participants complete a unit

Surveys

Pre/post surveys in workshop

Evaluating Team

Ordinal data

FWL project manager and staff, potential investors, potential clients FWL project manager and staff, potential investors, potential clients FWL project manager and staff, potential investors, potential clients FWL project manager and staff, potential investors, potential clients

Participants complete Unit 1 (10-15 hours)

Tests

Pre/post tests in workshop

Evaluating Team

Range of scores

Participants complete Unit 2 (12-18 hours) Participants complete Unit 3 (10-15 hours)

Structured Interview

Pre/post interviews in workshop

Evaluating Team

Scaled items

Open ended Pre/post tests tests in workshop involving participants writing objectives

Evaluating Team

Objectives scored by a rubric

Christina Narayan | EDTECH 505-4172

Task Schedule Task 1. Meet with FLW Project Manager and staff to revisit proposal and discuss details. 2. Provide IC with all DIP training materials. 3. Submit data-collection plan and all drafts of scoring guides, rubrics, report templates, tests, surveys and interview questions. 4. Provide IC with feedback on data collection plan and drafts of all instruments. 5. Revise data collection plan and drafts accordingly and provide FLW with all final copies. 6. Primary Goal: Submit all reports from the data sources for the Business Viability Model for review and present findings. 7. Determine if Secondary Goal should be evaluated based on the Business Viability Model 8. If continuing with evaluationSecondary Goal: Start group of participants in Unit 1. (Evaluation process includes pre/post surveys and pre/post tests) 9. Summarize data findings to FLW for Unit 1. 10. Start group of participants in Unit 2. (Evaluation process includes pre/post structured interviews) 11. Summarize data findings to FLW for Unit 2. 12. Start group of participants in Unit 3. (Evaluation process includes openended pre/post tests and end surveys) 13. Summarize data findings to FLW for Unit 3. 14. Report summary and final reports for primary and secondary goals and recommended next steps. Responsible Agency IC Deadline Date April 1, 2012

FLW IC

April 1, 2012 May 1, 2012

FLW

May 15, 2012

IC

June 1, 2012

IC

August 1, 2012

FLW

August 15, 2012

IC

August 22, 2012

IC IC

September 7, 2012 September 14, 2012 October 1, 2012 October 8, 2012

IC IC

IC IC

October 22, 2012 November 1, 2013

Christina Narayan | EDTECH 505-4172

Evaluation Proposal

Project Personnel Meet the evaluation team at Integrity Consulting: D. Lee Manning is the senior level evaluation specialist for Integrity Consulting. Mr. Manning will lead the day to day implementation of the evaluation, including development of the data collection instruments, research and data collection, supervision of the team, editing required reports, presenting the findings liaising with FWL. Mr. Manning holds a B.A. in Business Administration from Brigham Young University and a Masters in Educational Technology from San Diego State. Arvin V. Narayan will manage collecting, summarizing and interpreting the data, writing the required reports and developing the presentations of the findings. Mr. Narayan has a B.S. in Communications from California State and a Masters in Public Policy from the University of California at Berkeley.

Julina M. Egbert is the graduate assistant from the University of California at Berkeley, supporting the team at every level. Budget Personnel salaries: $44,625 D. Lee Manning: 50 days @ $400/day Arvin V. Narayan: 60 days @ $275/day Julina M. Egbert: 65 days @ $125/day $20,000 $16,500 $8,125

Travel: $180 Estimated travel mileage: 600 miles @ .30/mile (40 miles round trip from IC in Berkeley to FLW in Sausalito) Supplies, Materials, Copies: $700 Supplies: Copies: Total Budget: $45,505 Notes: This budget is based on full evaluation process. FLW will provide facilities for all workshops provided for participants as part of the evaluation process.
a

$200 $500

Reference for the Business Viability Model retrieved from http://bestentrepreneur.murdoch.edu.au/Understanding_Dimensions_of_Business_Viability.pdf Christina Narayan | EDTECH 505-4172