!"#"$!%& (!

)#("%*+#

Research Prospectus

University of Kentucky

!"#"$!%& (!)#("%*+#

Need for the Study Online courses have seen considerable growth over the past 10 years. An annual survey by Babson Survey Research Group found over 6.1 million students took at least one online class during the fall 2010 semester, a 10.1 percent increase over the year before (Allen & Seaman, 2011, November). This is significantly different than in Fall 2002, when enrollment was 1.6 million, indicating an increase in enrollment of over 380 percent. Current research in the area of distance education and online learning has focused on attitudes, student satisfaction, and final course grades (Walker & Fraser, 2005). However, there is little research to date on the quality of courses as gauged by a national benchmark. Given the growth in online course offerings, a consistent quality measure could enable instructors to concentrate on engaging the student in the learning process, and better support students in meeting the outcomes of the course. Having high quality online courses allows instructors to avoid common pitfalls of poorly designed materials and design. For example, poor course quality can require instructors to spend time troubleshooting problems, clarifying materials, and providing supplemental course materials (Hirumi, 2005) rather than enhancing the course with active learning activities and monitoring the learning activities of students. Having a poor course design can also result in online instructors feeling overwhelmed and frustrated with the online learning process. Using documented online course quality indicators throughout the design, delivery, and evaluation can increase the understanding and overall effectiveness of the instructor, but also support the student in navigating the course and completing assignments. Given the paucity of research and a need for consistent course quality and academic integrity, a thorough comparison of current online course quality is essential. The purpose of this

!"#"$!%& (!)#("%*+#

study will be to add to the research on course quality measures and standards for online and distance education within the context of higher education. Research Questions Purpose of the Study This study will be designed to determine the extent to which Kentucky Community and Technical College (KCTCS) faculty utilize quality standards during the design and continuous improvement of their online courses. The following research questions will guide the study. 1. To what degree does the KCTCS Learn on Demand Online Quality Assurance Rubric align with national online course quality standards? 2. To what degree do KCTCS online courses meet quality standards? Specifically, a. How do faculty rate their courses on quality measures as outlined in the KCTCS Learn on Demand Online Quality Assurance Rubric? b. Do ratings differ across faculty groups and program types? c. To what degree do faculty utilize quality standards for design, development, and continuous improvement of Learn by Term online courses? Proposed Methodology This study will use a descriptive design to gather information on the current status of the use of quality measures in the design, development and continuous improvement of online courses within the KCTCS system. The study will be submitted to both the University of Kentucky Institutional Review Board (IRB) and KCTCS IRB for approval. To answer research question 1 (i.e., degree KCTCS Learn on Demand (LOD) Online Quality Assurance Rubric aligns with national quality standards) quality measures outlined in the KCTCS LOD rubric will be compared to standards identified through six national studies. These

!"#"$!%& (!)#("%*+#

studies include (a) Institute for Higher Education Policys Quality on the Line (Phipps & Merisotis, 2000), (b) Marshalls Quality Framework for Continuous Improvement (Marshall, 2010), (c) Sheltons Quality Scorecard (Shelton, 2010), (d) Commission on Colleges Southern Association of Colleges and Schools Best Practices (Commission on Colleges Southern Association of Colleges and Schools, 2000, December), (e) Moores Sloan Consortium Quality Framework (Moore, 2005), and (f) Southern Regional Education Boards Standards for Quality Online Courses (Southern Regional Education Board, 2006). The degree of alignment will be calculated and reported across standards. Research question 2 (a-c; i.e., degree to which KCTCS online courses meet quality standards) will be addressed via survey. Survey questions will include how faculty designed courses, use of national or KCTCS standards during design, re-evaluation of courses for continued improvement and tools used for this evaluation. They will also be asked questions about how they perceive course quality throughout KCTCS. In addition, faculty will be asked to self-rate one course via a modified quality rubric based on the KCTCS Learn on Demand rubric and national benchmarks. These data will be used to describe course quality in general and across the programs and demographics of the courses and faculty. To increase validity, 4-5 courses will be chosen at random from each program area (N = 8), rated by the researcher, and compared to faculty ratings. Prior to implementation, cognitive and/or pilot testing will be conducted on the survey. This process will enhance the reliability of the data and ensure the survey delivery is efficient and smooth, survey directions are clear, and easy to read.

!"#"$!%& (!)#("%*+# Research Setting/Context

Like many other colleges and universities, KCTCS began developing online courses quickly with little to no training in best practice for online teaching. Faculty designing the earliest online courses may yet to have received training in best practice. All new faculty who teach online are required to complete training on best practice, course quality, and the KCTCS learning management system (LMS). There are no system-wide quality measures for KCTCS Learn by Term (LBT) courses. Each college is responsible for ensuring the quality standards of the LBT courses and the faculty who teach them are in place. For this study, online learning is defined as an educational process where students are separated by distance and time from one another and from the instructor (Andrews & Tynan, 2012; Keegan, 1996; Rekkedal, 2004). Learning content and instruction are delivered through computers and computer networks. Face-to-face interactions may occur through two-way communication via computer networks, but are not required to complete the learning process. Any interaction between learners and between learners and teachers is mediated by technology. KCTCS Learn by Term (LOT) is defined as courses which are offered online and follow the traditional course schedule (Kentucky Community and Technical College System, n.d.). The traditional schedule is based on a semester system and follows the start and end dates for other on-campus courses. KCTCS Learn on Demand (LOD) is defined as online and hybrid courses which follow an open entry/open exit course model. In the open entry/exit model, students may start a course at any time of the year regardless of the local colleges regular course calendar. Students have the opportunity to complete the course early, but also have a specific amount of time in which they must complete the course.

!"#"$!%& (!)#("%*+#

Research Sample, Data Sources and Analysis The studys sample will include the population of KCTCS faculty teaching at least one online course. According to data collected through PeopleSoft (the KCTCS college management software) the approximately 1500 KCTCS faculty were teaching online in the current semester (spring 2014). To ensure representativeness of data collected, a sampling frame will be used that includes KCTCS site (e.g. Jefferson, Western Kentucky, and Bluegrass), academic area (e.g. written communication, oral communication, and technical), and faculty gender. An online survey will be sent to all KCTCS faculty teaching online classes. This list of faculty members will be gathered from the distance learning coordinator at each KCTCS college. Based on the respondents after the first round of surveys, additional recruitment efforts will be initiated to address gaps based on the sampling frame. As needed, KCTCS distance learning coordinators from each college will be contacted and asked to support additional recruitment efforts. Descriptive statistics will be used to examine online courses within the KCTCS system. Descriptive statistics were chosen as they support the simplification of large amounts of data into descriptive summaries. These summaries will allow data to be compared across courses and instructional areas (McMillan & Schumacher, 2009; Schreiber & Asner-Self, 2011). Measures of central tendency, variability, and t-tests will be used for comparative analysis across respondents (e.g. gender, KCTCS sites, and academic area). I will also use inter-rater reliability to ensure reliability.

!"#"$!%& (!)#("%*+#

References Allen, I. E., & Seaman, J. (2011, November). Going the distance: Online education in the United States, 2011: BABSON Survey Research Group. Andrews, T., & Tynan, B. (2012). Distance learners: Connected, mobile and resourceful individuals. Australasian Journal of Educational Technology, 28(4), 565-579. Commission on Colleges Southern Association of Colleges and Schools. (2000, December). Best practices for electronically offered degree and certificate programs. Retrieved from http://www.sacscoc.org/policies.asp Hirumi, A. (2005). In search of quality: An analysis of e-learning guidelines and specifications. Quarterly Review of Distance Education, 6(4), 309-329. Keegan, D. (1996). Foundations of distance education. New York, NY: Taylor & Francis. Kentucky Community and Technical College System. (n.d.). What is KCTCS online?, from http://www.kctcs.edu/KCTCS_Online.aspx Marshall, S. (2010). A Quality Framework for Continuous Improvement of e-Learning: The eLearning Maturity Model. Journal of Distance Education, 24(1), 143-165. McMillan, J. H., & Schumacher, S. (2009). Research in education: Evidence-based inquiry: Pearson Education, Limited. Moore, J. C. (2005). The Sloan consortium quality framework and the five pillars. The Sloan Consortium. Retrieved July, 15, 2007. Phipps, R., & Merisotis, J. (2000). Quality on the line: Benchmarks for success in Internet-based distance education. Washington, DC: Institute for Higher Education Policy. Rekkedal, T. (2004). Internet based e-learning, pedagogy and support systems. In J. A. Brindley, C. Walti & O. Zawacki-Richter (Eds.), Learner support in open, distance and online learning environments (Vol. 9, pp. 71-94). Carl von Ossietzky University of Oldenburg: ASF. Schreiber, J., & Asner-Self, K. (2011). Educational research: The interrelationship of questions, sampling, design, and analysis. Danvers, MA: Wiley. Shelton, K. (2010). A quality scorecard for the administration of online education programs: A delphi study. (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses. (ATT 3423965) Southern Regional Education Board. (2006). Standards for Quality Online Courses: Southern Regional Education Board (SREB).

!"#"$!%& (!)#("%*+#

Walker, S., & Fraser, B. (2005). Development and validation of an instrument for assessing distance education learning environments in higher education: The distance education learning environments survey (DELES). Learning Environments Research, 8(3), 289308. doi: 10.1007/s10984-005-1568-3