Final Project 2 Final Project I ntroduction The Veterans Health Administration (VHA) is focused on providing cutting-edge care to Veterans. In order to provide top of the line care, the VHA has tasked itself with becoming a data-driven organization. The VHA realized more needed to be done to increase the analytics capability of their staff; they wanted to create an analytically competent workforce. In 2011 they created the Field Based Analytics Team (FBA) to promote consultation, education and tools relating to data analytics. The VHA partnered with Nebraska Methodist College in 2010 to create the Healthcare Analytics Certificate Program. The courses are non-credit and available only to employees of the Department of Veterans Affairs. Employees must complete all four courses to earn the certificate. Since the program was created, 1216 employees have enrolled, however only 233 have earned the certificate. The Field Based Analytics Team was tasked with increasing the amount of graduates from 19% to 30% in the next year. Course Enrolled & Passed Course 1 Healthcare Data Analysis Offered 11 times: 1216 enrolled and 983 passed the course Course 2 Healthcare Systems Improvement Offered 8 times: 677 enrolled and 616 passed the course Course 3 Healthcare Data Mining Offered 5 times: 328 enrolled and 258 passed the course Course 4 Healthcare Project Analysis Offered 4 times: 245 enrolled and 233 passed the course
The FBA team expressed concerns with the statistical knowledge participants had prior to entering the course. They want to create a module that would be a pre-requisite to the HACP, focusing on knowledge that is necessary in order for the participants to be successful in all four Final Project 3 courses. The FBA team and the Instructional Designer decided on the following instructional goal: Participants will review 5 statistical terms (maximum, minimum, range, mean, and medium), use them in an Excel worksheet example, and then apply the 5 statistical terms to their own data set Analysis As stated in the introduction, the participants of HACP are all VHA employees. Employees must have a bachelors degree and supervisor approval to enroll in the course. At the end of each course, a survey is conducted which the Instructional Designer used to further understand the needs of the course. Employees range in age from 26 to 64 and are in clinical and non-clinical positions within the VHA. Most employees identify themselves as being good in math, however 56% state they are comfortable with statistics. Employees are asked the likelihood of taking the next course and below are the response: *71% of Course 1 participants were likely to take Course 2 *49% of Course 2 participants were likely to take Course 3 *98% of Course 3 participants were likely to take Course 4 The Instructional Designer administered another survey to students who passed courses 1, 2, or 3 and asked if they were continuing to the next course, why: Wanted to complete the program and obtain the certificate Found the information useful in their job The students were then asked if they passed a course, however did not take the next course why not: Didnt have the time, work was too busy Final Project 4 Course was much harder than expected because of the math Context All employees have a designated computer that is loaded with the Microsoft Suite (Excel, Word, PowerPoint, etc.). Computers are located in both public and private locations depending if the employee is clinical or non-clinical. Employees have the ability to print from their workstation to any printer in the facility. Each facility also has a minimum of one computer lab that serves as a training room. Each training room is equipped with a minimum of 15 computers, whiteboards, projector w/screen, and whiteboards. Training rooms are reserved through a reservation management system which is fairly easy to utilize. The facilitator will conduct the training in a classroom, however the training will be set- up as self-paced learning for employees who are located in remote stations. Each participant will have their own computer with access to Microsoft Excel.
Main Instructional Goal: Participants will review 5 statistical terms (maximum, minimum, range, mean, and medium), use them in an Excel worksheet, and choose the correct term in a knowledge check.
1. Review 5 statistical terms (maximum, minimum, range, mean, and medium) 2. Use them in an Excel worksheet 3. Apply 5 statistical terms to own dataset Final Project 5 Subordinate Skills Analysis
Final Project 6 Objectives and Assessment
Main Instructional Goal: Participants will review 5 statistical terms (minimum, maximum, range, mean, and medium), use them in an Excel worksheet, and choose the correct term in a knowledge check.
Terminal Objective: When presented with the terms minimum, maximum, range, mean and medium, employees will know what each term means. Employees will know how to use Excel to calculate each term using any data. Employees will apply the 5 statistical terms to their own existing worksheet. Main Step in I nstructional Goal Objective 1.0Review the statistical terms: minimum, maximum, range, mean and median. When asked to identify the most common statistical terms, employees should identify minimum, maximum, range, mean and median. Subordinate Skill Objective 1.1 Define the term minimum Given the term minimum, maximum, range, mean, and median employees will define each term as follows: Minimum as the smallest number in a data set. Maximum as the largest number in a data set. Range is the difference between the minimum and maximum. Mean is the average of the data set Median is the middle number when the data is ordered from smallest to largest. If there is an odd number of data points, the median will be the middle data point. If there is an even number of data points, the median is the average of the two middle points.
1.2 Define the term maximum 1.3 Define the term range 1.4 Define the term mean 1.5 Define the term median 1.6 Identify why these 5 statistical terms are useful in data analysis When asked to describe why these terms are useful in data analysis, employees will describe spread of the data, proximity of mean and median, and distribution of data. Main I nstructional Goal Objective 2.0 Demonstrate the 5 statistical terms in Excel Given a computer with Microsoft Excel, calculate the minimum, maximum, range, mean and median for an example worksheet. Subordinate Skill Objective 2.1 Open Excel File Using a computer with Windows 7, open Microsoft Excel and access the workbook Stats 5 Terms 2.1.1 Click on the Office button located in the lower left corner of the screen 2.1.2 Click on All Programs Final Project 7 2.1.3 Click on Microsoft Office 2010 2.1.4 Click on Microsoft Excel 2010 2.1.5 Click on the File tab in the top left corner 2.1.6 Click on Open 2.1.7 Click on Desktop 2.1.8 Click on Stats 5 Terms NOTE: See Addendum 1 for copy of worksheet Subordinate Skill Objective 2.2. Enter 5 statistical terms Using the workbook Stat 5 Terms, type the terms: Min, Max, Range, Mean and Median to prepare worksheet for calculation of terms 2.2.1 In cell F7 type the word Min 2.2.2 In cell F8 type the word Max 2.2.3 In cell F9 type the word Range 2.2.4 In cell F10 type the word Mean 2.2.5 In cell F11 type the word Median Subordinate Skill Objective 2.3 Calculate Minimum Using the Excel workbook Stat 5 Terms, calculate the Minimum for the data set in cells D4-D15 2.3.1 Click on cell G7 2.3.2 Click on the Formulas tab in the Ribbon 2.3.3Click the arrow below AutoSum 2.3.4 Click on Min 2.3.5 Left click on cell D4 and drag to cell D15 2.3.6 Hit Enter on keyboard Subordinate Skill Objective 2.4 Calculate Maximum Using the Excel workbook Stat 5 Terms, calculate the Maximum for the data set in cells D4-D15 2.4.1 Click on cell G8 2.4.2 Click on arrow below AutoSum 2.4.3 Click on Max 2.4.4 Left click on cell D4 and drag to cell D15 2.4.5 Hit Enter on keyboard Subordinate Skill Objective 2.5 Calculate Range Using the Excel workbook Stat 5 Terms, calculate the Range for the data set in cells D4-D15 2.5.1Click on the cell G9 2.5.2 Hit the = on keyboard 2.5.3 Type G8-G7 2.5.4 Hit Enter on keyboard Subordinate Skill Objective 2.6 Calculate Mean Using the Excel workbook Stat 5 Terms, calculate the Mean for the data set in cells D4-D15 2.6.1 Click on cell G10 2.6.2 Click the arrow below AutoSum 2.6.3 Click on Average Final Project 8 2.6.4 Left click on cell D4 and drag to cell D15 2.6.5 Hit Enter on keyboard Subordinate Skill Objective 2.7 Calculate Median Using the Excel workbook Stat 5 Terms, calculate the Median for the data set in cells D4-D15 2.7.1 Click on cell G11 2.7.2 Click on Insert Function 2.7.3 In the Search for a Function box, type Median and click on Go 2.7.4 The term will appear at the top in the Select a function box 2.7.5 Click OK 2.7.6 The Function Arguments box will appear; erase anything in the Number 1 field 2.7.7 In the Number 1 field, type in D4:D15 2.7.8 Click OK Main I nstructional Goal Objective 3.0 Apply 5 statistical terms to own data set When asked to apply the 5 statistical terms: maximum, minimum, range, mean and median employees will use a data set of their own to calculate the 5 terms using Excel. Subordinate Skill Objective 3.1 Calculate maximum, minimum, range, mean, and median Using the employees Excel workbook, the employee will correctly calculate the 5 statistical terms on the worksheet with their identified data set. 3.1.1 Open employees existing worksheet in Excel 3.1.2 Type the words: minimum, maximum, range, mean, and median somewhere on the worksheet 3.1.3 Next to each of the 5 statistical terms, correctly calculate each term based on the employees data set
Assessment Performance Objectives for Subordinate Skills Parallel Test I tem 1.0When asked to identify the most common statistical terms, employees should identify maximum, minimum, range, mean, and median. Below is a list of statistical terms. Circle the 5 terms that you feel are the most common statistical terms that might be used in day-to- day analysis.
1. Z-test 2. Box model Final Project 9 3. Range 4. Chi square statistic 5. Mean 6. Maximum 7. Margin of Error 8. Minimum 9. Median 10. Ordinal variable 1.1 1.5Given the term maximum, minimum, range, mean, and median employees will define each term as follows: Minimum as the smallest number in a data set. Maximum as the largest number in a data set. Range is the difference between the minimum and maximum. Mean is the average of the data set Median is the middle number when the data is ordered from smallest to largest. If there is an odd number of data points, the median will be the middle data point. If there is an even number of data points, the median is the average of the two middle points.
Using the provided job-aid, review the definitions of each term.
Using the data set below, write the correct data next to the term.
2.1 2.1.8 Using a computer with Windows 7, open Microsoft Excel and access the workbook Stat 5 Terms Once the exercise is complete, the facilitator will check the worksheet to ensure the answers are correct. To be successful you must have 7 out of 7 correct.
1. Opened workbook Yes No 2. Typed all 5 terms Yes No 3. Calculated min Yes No 4. Calculated max Yes No 5. Calculated range Yes No 6. Calculated mean Yes No 7. Calculated range Yes No 2.2 2.2.5 Using the workbook Stat 5 Terms, type the terms: Min, Max, Range, Mean and Median to prepare worksheet for calculation of terms 2.3-2.3.6Using the Excel workbook Stat 5 Terms, calculate the Minimum for the data set in cells D4-D15 2.4-2.4.5 Using the Excel workbook Stat 5 Terms, calculate the Maximum for the data set in cells D4-D15 2.5-2.5.4 Using the Excel workbook Stat 5 Terms, calculate the Range for the data set in cells D4-D15 2.6-2.6.5 Using the Excel workbook Stat 5 Terms, calculate the Mean for the data set in cells D4-D15 2.7-2.7.8 Using the Excel workbook Stat 5 Final Project 10 Terms, calculate the Median for the data set in cells D4-D15 3.1 3.1.3 Using the employees Excel workbook, the employee will correctly calculate the 5 statistical terms on the worksheet with their identified data set. Once the exercise is complete, the facilitator will check the worksheet to ensure the answers are correct. To be successful you must have 7 out of 7 correct.
1. Opened workbook Yes No 2. Typed all 5 terms Yes No 3. Calculated min Yes No 4. Calculated max Yes No 5. Calculated range Yes No 6. Calculated mean Yes No 7. Calculated range Yes No
Final Project 11 Copy of Assembled Assessment
Calculate 5 Common Statistical Terms Review the 10 terms below. Based on your existing knowledge of math and statistics, identify the 5 most common statistical terms.
1. Z-test 2. Box model 3. Range 4. Chi square statistic 5. Mean 6. Maximum 7. Margin of error 8. Minimum 9. Median 10. Ordinal variable
Using the data set below, write the correct data next to the term.
Final Project 12
Using the instructions provided, calculate the answers for the following 5 statistical terms: minimum, maximum, range, mean, and median. You will practice using the provided workbook Stat 5 Terms. To successfully complete this class, you will use your own workbook and accurately calculate the 5 statistical terms for your data. The facilitator will review your answers for accuracy. Practice Workbook Stat 5 Terms 1. Opened workbook Yes No 2. Typed all 5 terms Yes No 3. Calculated min Yes No 4. Calculated max Yes No 5. Calculated range Yes No 6. Calculated mean Yes No 7. Calculated range Yes No
Personal Workbook 1. Opened workbook Yes No 2. Typed all 5 terms Yes No 3. Calculated min Yes No 4. Calculated max Yes No 5. Calculated range Yes No 6. Calculated mean Yes No 7. Calculated range Yes No Final Project 13
I nstruction Employees will be given the job-aid to use for both facilitator and self-paced training versions. The job-aid reviews the definitions of the 5 statistical terms, reviews why they are important, provides an example, and then instructions on how to calculate each of the terms using Excel. If the class is facilitator led, its recommended that they demonstrate this information via projection from computer. Employees will complete the course using a computer; therefore if the course is instructor led they will be able to follow along as the instructor provides the examples.
Final Project 14
Final Project 15
Final Project 16 Evaluation Topic: Calculating 5 Statistical Terms Designer: Melinda Dye Criteria Excellent Good Acceptable Needs Improvement Score Analysis Optimal analysis; they possessed the necessary skills, opportunities existed to make choices, there was varying degree of effort, and modification of instruction was possible if needed Positive analysis; possessed most of the skills, there were a few opportunities to make choices, some varying degree of effort required, and some modification of instruction existed. Neutral analysis; possessed a few of the skills needed for training, there was at least one choice during instruction, and there was little modification possible. Poor analysis; some skills were not present for training; no choices during instruction, and no modification possible. 3 Design Optimal design; there were a wide-variety of activities, the objectives reinforced the behaviors, multiple opportunities for mastery of goals was possible, and varied instructional content. Positive design; there were multiple activities, most objectives reinforced the behaviors, some opportunities for mastery of goals, and some varied instructional content existed. Neutral design; There were 1-2 activities, at least one objective reinforced the behaviors, a few opportunities for mastery of goals, and little varied instructional content. Poor design; There were no activities, objectives did not reinforce behaviors, no opportunity for mastery of goal, and no variance of instructional content. 4 Development Optimal development: Course includes multiple options for modeling, activities reinforce usefulness of tasks, activities support scaffolding, collaborative learning is included, students and instructions able to set short and long term goals. Positive development: Course includes a few options for modeling, some activities reinforce usefulness of tasks, a few activities suppose scaffolding, possibility of collaborative learning is included, and students and instructors are able to set long or short term goals. Neutral development: Couse includes one options for modeling, activities might reinforce usefulness of tasks, activities possibly support scaffolding, collaborative learning is limited, and students or instructors are able to set short or long term goals. Poor development: Course does not including modeling, activities do not reinforce usefulness of tasks, scaffolding is not included, collaborative learning is not includes and goal- setting is not available. 3/2 Implementation Optional implementation: Instructor provides ability feedback when opportunity exists, possesses a strong level of enthusiasm and interest, classroom environment is positive and rewards are established for meeting goals. Positive implementation: Instructor provides varying some ability feedback, has enthusiasm and interest, classroom environment is positive most of the time, and there are limited rewards. Neutral implementation: Instructors provides ability feedback occasionally, has enthusiasm or interest, classroom environment is positive sometimes, and rewards could be offered. Poor implementation: Instructor provides no ability feedback, has little or no interest or enthusiasm, classroom environment is not positive or negative and there is no option for rewards. 3/2
Final Project 17 I nstructional Designer Reflection of Motivation Use in I nstruction The need for a course on the 5 statistics of minimum, maximum, range, mean and median was identified during the last year upon evaluating the course completion rates of the Healthcare Analytics Certificate Program (HACP). The instructors of the HACP have provided feedback when asked about the decrease from course to course and their main feedback tends to be on the lack of knowledge regarding common statistical terms. Many of the participants were exposed to these terms in past educational experiences, however are not familiar enough with them now. The instructors stated that the lack of knowledge with these terms can be a challenge for students and cause them to spend more time than expected on homework. The instructional designer used the knowledge provided by the instructors to create the mini course on the 5 statistics. Since the HACP is voluntary, enrollment indicates some level of learner motivation. It would be logical to assume since the learner is motivated that they would seek out independently to understand these terms. In some cases this would be true, however some students over-estimate their knowledge of statistics, and some students dont have the time to do further research and stay current with homework. Providing a course that could be taken during the work schedule and will have little interruption to their workday will motivate some to take the course. All of the students attending this course will have the necessary skills of being able to operate a computer and use Excel. A prerequisite of HACP is a basic understanding of Excel. The course will provide learners with the opportunity to make a choice in what data they use for the final exercise; learners could select a data set theyve been tasked with analyzing or a random sample. The instructors are experts in the subject and can provide additional examples if needed based on classroom feedback. Final Project 18 The course was designed with mastery in mind; learners will leave the class easily knowing how to calculate the 5 terms with or without the use of Excel. Learners will complete a variety of exercises which will reinforce the 5 terms: a hardcopy example with their job-aid, an example with a provided data set in Excel, and a final exercise using their own data set in Excel. Learners will complete a variety of activities such as identifying the 5 statistical terms, reviewing the definitions, hardcopy practice and two Excel examples. The use of the job-aid will provide a reference for use during class as well as after class. Using a job-aid will also eliminate any anxiety with learners; in addition the course is not graded or used for any type of performance evaluation. Learners work will be evaluated, however its used only to ensure correct calculations; again, this should not invoke anxiety since they are not being scored or used for performance evaluation. Learners will have the opportunity to observe the instructor perform the calculations by projecting their Excel workbook on to a big screen to be observed by everyone. The use of examples and exercises will provide learners the opportunity to receive feedback throughout class. There are no rewards for this course and only one goal learn how to calculate the 5 statistics. The motivation for this course is intrinsic in nature; learners want to do well in the HACP, and to do well they need this course. Completing all four courses will earn the certificate which could improve their performance evaluation and lead to extrinsic motivation which is a pay increase. Evaluation The opportunity was not available to test Calculate 5 Common Statistical Terms with V.H.A. employees, therefore test 3 adult users were selected who currently use computers, Excel, and might have a need for using minimum, maximum, range, mead, and median. A Final Project 19 classroom was not available, however each user did have their own computer. Each participant received a copy of the job-aid and the class was led by the instructional designer. Users did not complete the last exercise which was to use an existing data set of their own.
Question User 1 User 2 User 3 Review the 10 terms below. Based on your existing knowledge of math and statistics, identify the 5 most common statistical terms. 1. Range 2. Mean 3. Maximum 4. Minimum 5. Median 1. Z-test 2. Maximum 3. Minimum 4. Median 5. Mean 1. Range 2. Mean 3. Maximum 4. Minimum 5. Median Using the data set below, write the correct data next to the term. Min: 13 Max:56 Range: 43 Mean: 32.4 Median: 30 Min: 13 Max:56 Range: -43 Mean: 32.4 Median: 30 Min: 13 Max:56 Range: 43 Mean: Median:
Evaluation: Motivation Questionnaire Learners were given the following questionnaire after completing the course (except for the last exercise of using their own data set). The evaluation used Kellers ARCS model as a basis for development. At the end of the questionnaire, learners were able to provide optional comments about any aspect of the course. The results of the questionnaire are not completely accurate since the learners who tested the materials are not employees of the V.H.A. and do not have ability to enroll in the HACP.
Final Project 20
This questionnaire will provide feedback on the effectiveness of the materials for the course Calculate 5 Common Statistical Terms. Please rate each question using the following scale: 1=poor 2=somewhat poor 3=neutral 4=somewhat good 5=good. At the end of the questionnaire is an area for comments or explanation of any answer. Question Learner 1 Learner 2 Learner 3 Attention: How well did the instructional activities hold your interest? 1. Identifying the 5 statistical terms 2. Calculating the 5 terms using the provided data set on hardcopy 3. Watching the instructor demonstrate how to calculate in Excel 4. Using the existing data set to calculate in Excel 5. Using my own data set to calculate in Excel
5 4
N/A
5 N/A
5 5
N/A
5 N/A
5 4
N/A
5 N/A Relevance: To what degree do you think the course activities were relevant to your job? 1. Defining the 5 statistical terms 2. Calculating the 5 statistical terms in Excel
5 3
5 2
5 5 Confidence: To what degree do you have confidence in your ability to calculate the 5 statistical terms? 1. Min 2. Max 3. Range 4. Mean 5. Median
5 5 5 5 5
3 3 3 3 3
5 5 5 5 5 Clarity: To what degree of clarify do you have in the clarity of material? 1. Identifying the 5 statistical terms 2. Definitions of each term 3. Instructions for calculating in Excel