1

CALIFONIA STATE UNIVERSITY

MONTEREY BAY

UCAT Training Website

CAPSTONE PROPOSAL

Submitted in partial satisfaction of requirements of the degree of

MASTER OF SCIENCE in

Instructional Science and Technology

Bassel El Ahmadieh

7 December 2017

Capstone Approvals: (At least one advisor and capstone instructor should approve)

___________________________ _____________
Advisor Name Signature Date

__________________________ _____________
Capstone Instructor Name Signature Date
 2

Table of Contents

Executive Summary4

Introduction and Background.4

Problem description.......5

Target Audience..6

Setting. .......6

Literature Review................................................................................................................7

Solution ..............................9

Goals..........9

Learning objectives ......10

Proposed solution......11

Learning Theories.........11

Principles...12

Strategy and Role of the trainer....12

Delivery system and Media decisions...13

Challenges........14

Method...14

Deliverables.... .15

Resources......16

Time and Cost ......16

Timeline........17

Evaluation and Testing......18

Formative Evaluation....18
 3

Usability Testing..18

Pre- and Post-Testing.......21

Results.......21

Summative evaluation..22

Conclusion......23

References.25

Appendices.....27

Appendix A...27

Appendix B...29

Appendix C...31

Appendix D...33

Appendix E...35

Appendix F...37

Release Form...39
 4

Executive Summary
UCAT Training Website trains UMA teachers into how to facilitate the integration
of Universal Curriculum Assessment Tool (UCAT) into teaching Levantine dialect by
developing and conducting professional development online sessions on infusing UCAT
into the core areas of the academic curriculum. This provides insight on ways to increase
teachers use of UCAT as it positively impacts students performance in the classroom.
Using UCAT in the classroom keeps students alert and eager to learn. Plenty of
studies show that technology has an impact on the instructional process since it stimulates
students thinking and makes them engage in the learning process. UCAT helps students
create, analyze and use critical thinking which makes learning more fun and meaningful to
their daily life experience. Moreover, The UCAT Training Website consists of pre-test,
modules, post-test and survey
Module 1: UCAT Terminology; Module 2: Creating listening and FLO activities on
UCAT; Module 3: Creating reading activities on UCAT; Module 4: Design a lesson on
UCAT.
Ten foreign language teachers who are teaching Levantine dialect, a spoken Arabic
of Lebanon, Syria, Jordan, and Palestine, volunteered to take this training. The teachers
worked on this training during their spare time at work. The foreign language teachers first
took the pre-test in order to be tested on their information about UCAT terminology, UCAT
features, development of listening, FLO and reading activities. These questions look the
same as the final test in UCAT module. The foreign language teachers took the post-test
after the training which is exactly the same as the pre-test.
Teachers who took the final evaluation survey that was hosted on Google Forms
provided effective feedback. The feedback from the foreign language teachers was collected
and analyzed for the purpose of improving the UCAT Training Website.

Introduction and Background

Technology in the classroom assists in students autonomous learning and students
perform inquiry-based instructional activities and a variety of other tasks (Roschelle & Pea,
2002). The presence of technology may help students excel more in the learning process and
 5

enable teachers to reach all the students. Technology helps teachers create instructional
strategies that address students various learning needs. For instance, teachers try to engage
students by preparing visual representations for instruction such as PowerPoint presentations
containing pictures, charts, graphs, maps or videos to assist the visual learner in grasping
better the information. Thus, teachers can individualize instruction by choosing materials
that benefit a variety of academic levels and learning preferences. Since technology plays a
major role in bettering student learning experiences, improving learner productivity and
enabling teachers to deliver the instruction effectively, I designed and developed Universal
Curriculum Assessment Tool (UCAT) Training Website that trains teachers at the Defense
Language Institute (DLIFLC) how to use UCAT to engage students in the learning process
and motivate them to achieve better results. The UCAT Training Website includes
instructional training into how to use UCAT in teaching DLIFLC students a second
language. Moreover, the UCAT Training Website instructs teachers by showing tutorials on
how to use this powerful tool to create, develop and design activities, tasks, quizzes,
assignments, and lessons to increase students proficiency.
Problem description
The majority of teachers at our school are not using UCAT in the teaching process
simply because they dont know how to use this powerful tool. UCAT is an online
curriculum delivery system specifically tailored to meet the needs of DLIFLC faculty and
students. It allows foreign language teachers to develop and design online interactive
activities that engage students. The format of UCAT activities include categorization, true or
false, multiple choice, drag and drop, fill in the blanks, short answer, transcription and
matching. Teachers have expressed their desire to learn about this valuable tool since this
will assist in solving many training problems and aid in achieving their goals. Some teachers
believe that lack of using technology in the classroom may lead to having less interested
students who do not pay attention in the classroom and do not study at home. Moreover, the
needs assessment indicates that UCAT is a good training tool, and if used wisely can assist
in solving many training problems. This will ultimately increase student success rates.

Teachers and students at Defense Language Institute (DLI) can use UCAT in the
training process to increase Institute annual productivity. Nevertheless, the majority of
teachers dont benefit from this tool that may help students pass the Defense Language
 6

Proficiency Test (DLPT), which is an exam that measures students second language skills
in listening and reading. To learn more about this issue, I conducted an interview with the
chairperson asking questions about the technology tools and software applications that the
teachers use inside and outside the classroom. Moreover, I prepared a questionnaire to be
answered by the teachers in our department for the purpose of studying and analyzing how
the teachers use technology tools inside and outside the classroom to increase students
proficiency.

Target Audience

The audience is adults whose ages range between 30 and 80 years old. The gender
percentage is as follows: 55% females and 45% males. The audience is foreign language
instructors teaching Levantine dialect, a version of Arabic spoken by people of Lebanon,
Syria, Jordan, and Palestine, at DLIFLC. The audience is originally either from Lebanon,
Syria, Jordan, Iraq or Palestine living in the United States of America as permanent
residents or American citizens or holders of work visas. The data collected from the survey
shows that the majority of the teachers at our school are interested in learning about UCAT.
The teachers want to try UCAT because this will allow them to implement new technology
in the classroom and to develop activities that promote critical thinking and autonomous
learning. Employing UCAT will engage students in the learning process and help them
excel in their learning to achieve good results at the final exams.

The challenges that I might face when conducting training to the foreign language
teachers are that some teachers still have difficulties in learning new technology tools and
others feel that learning new software applications is wasting time because there is no time
to implement these tools in the classroom.

Setting
DLIFLCs goal in 2017-2018 is to get students pass the DLPT reading and listening
exam at a level 2+. The DLIFLC administrations goal for the Oral Proficiency Interview
exam (OPI) is level 2. Solving the motivation problem and finding a means to improve the
type and accessibility of instruction pave the road to more language training. To reach
DLIFLCs goals and to assist in solving training problems, DLIFLC administration
 7

distributed in December of 2013 iPads and MacBooks to teachers and students to be used
inside and outside the classroom. However, it is not enough to distribute iPads and
MacBooks to teachers and students. It is essential to continuously train teachers on how to
use these powerful tools in the training process. For instance, there are many benefits when
implementing UCAT in the learning process since this platform acts as an effective
instructional tool in the hand of teachers that promote students motivation, cooperation,
communication, and organization. In addition, UCAT assists in students autonomous
learning and students perform inquiry-based instructional activities and a variety of other
tasks. With the presence of UCAT, students excel more in the learning process and teachers
are able to reach all the students. Thus, teachers can individualize instruction by choosing
materials that benefit a variety of academic levels and learning preferences. Furthermore,
this can aid the teacher in delivering the information more easily and solving the problem
faster.
UCAT Training Website trains foreign language teachers in how to use and
implement UCAT in teaching a second language. UCAT offers a wide variety of elements
and possibilities for making the authentic materials more engaging and educational. UCAT
can help increase class participation by giving students more confidence and helping them
improve their listening, speaking, reading and writings skills.
The UCAT Training Website contains five sections. The first module includes a
module that introduces UCAT Terminology. The second module instructs foreign language
teachers on how to use UCAT to create online listening and FLO activities for students. The
third module demonstrates and trains foreign language teachers in how to use UCAT to
develop and design reading activities on UCAT to improve students reading skills. The
fourth module instructs teachers on how to design an online lesson on UCAT. There is an
evaluative assessment at the end of each module that examines the trainees on the topic
discussed by asking different types of questions to ensure that the learners understand the
topic. The fifth section gives trainees the opportunity to evaluate the UCAT Training
Website by providing feedback to continuously improve this project.
Literature Review
Using online software application in the classroom keeps students alert and eager to
learn. Studies show that technology has an impact on the instructional process since it
 8

stimulates the students thinking and makes them engage in the learning process. Many
theorists have studied the relationship between education and technology; especially after
the positive effect that computers and software have to our lives. The online delivery system
helps teachers and students to perform better in the training process by motivating students
to excel in their learning and studies (Biggs, 2011). Use of the online interaction solves
many training problems that the teachers face in the classroom especially when it comes to
time and quality. Utilizing computerized applications in education provides cognitive
implications for reorganizing mental process and thus helps students in using their critical
thinking by planning and solving problems (Pea & Kurland, 1984). Moreover, online
interaction helps students to perform all language skills encouraging the collaborative
learning methodology in which students share information together via the internet (Berge
& Collins, 1995). The students work as a team to solve problems or to answer reading
comprehension questions, which encourage computer collaborative learning through
synchronous and asynchronous participation.
The online delivery system acts as an effective instructional tool in the hand of
teachers that promotes students motivation, cooperation, communication and organization
(Sherry, 1996). Technology in the classroom assists in students autonomous learning and
students perform inquiry-based instructional activities and a variety of other tasks (Hmelo-
Silver, 2004). With the presence of online interaction, students excel more in the learning
process (Palloff & Pratt, 1999). The online delivery system helps teachers to create teaching
strategies that address students learning needs. For instance, teachers try to engage students
by preparing interactive activities such as multiple choice, true or false, matching, and
categorization. Thus, teachers can individualize instruction by choosing materials that
benefit a variety of academic levels and learning preferences. Furthermore, the online
delivery system can aid the teacher in delivering the information more easily and solving the
problem faster (Mayer, 1998). Teachers using online interaction are able to create, design,
and develop activities that address students various learning abilities and levels. In addition,
the online delivery system provides the teachers with the option of adapting lectures,
curriculum and activities to best meet the students learning needs (Harasim, 1995).
The online delivery systems encourage the use of collaborative learning
methodology inside and outside the classroom. Classrooms should become classrooms
 9

more characterized by computer-supported collaborative learning. Students will work

towards shared understanding in groups. Students will build joint representations of their
knowledge. To enhance understanding, students will point to, annotate, and use external
representations in diverse sense-making and discourse practices (Roschelle & Pea, 2002).
Using online learning in training is a new trend in instructional technology in which
it acts as effective instructional tools in the hand of teachers and learners that promote
learners motivation, cooperation, communication, and organization. With the presence of
online learning, learners excel more in the learning process and teachers are able to reach all
the students (Clark & Mayer,2016). Technology tools make it easy for teachers to create
teaching strategies that address learners various learning abilities and needs. For instance,
teacher designs and develops different online activities depending on learners interests,
needs, and cultural background by using various online software applications. Therefore,
online training is more effective if used wisely with the implementation of effective
teaching strategies that address various learners needs, abilities and interests.

Solution
Goals of the Project
The purpose of the UCAT Training Website is to train foreign language teachers
who teach Levantine dialect, a spoken language of Lebanon, Syria, Jordan, and Palestine,
how to use UCAT in teaching the second language that assists in increasing student
motivation and proficiency. This website instructs teachers how to develop level 2+ lessons
and activities which assist in training students to pass the DLPT reading and listening exam
at a level 2+. These levels are specified by the ILR scale, which has six levels ranging from
zero (no functional ability) to five (full professional proficiency). Figure (1) describes ILR
Levels 2, 2+, and 3 that specify what student should master in each skill to reach these
levels.
 10

Figure 1: ILR scale Table (Brau, Brooks, 2009)

Learning Objectives
The learning objectives of UCAT Training Module are as follows:
From memory, the foreign language teachers will be able to recognize UCAT
terminology with 100% accuracy by answering multiple choice questions
Given a software simulation of UCAT, the foreign language teachers will be able to
create a module on UCAT and share it with other colleagues with 100% accuracy.
Given a laptop and access to the internet, foreign language teachers will be able to
design level 2+ listening and FLO activities using the Universal Curriculum
Assessment Tool (UCAT) without making errors
Given a laptop and access to the internet, foreign language teachers will be able to
design level 2+ reading activities with the Universal Curriculum Assessment Tool
(UCAT) without making errors
Given a laptop and access to the internet, foreign language teachers will be able to
design a level 2+ Lesson on Universal Curriculum Assessment Tool (UCAT)
without making errors
 11

Proposed Solution

The UCAT Training Website uses a multitude of different teaching modalities that
incorporate the various aspects of Captivate 9 during instructional training focusing on
learners different learning needs, levels, and abilities. UCAT Training Website demonstrates
step by step how to use this powerful tool inside and outside the classroom. The UCAT
Training Website enables teachers to learn about UCAT terminology, features and activity
components, and steps required for registering a UCAT account. Moreover, this Website
instructs foreign language teachers into how to create and design online UCAT activities and
lessons that target student needs to ensure the success of the students when taking the DLPT
and OPI. The UCAT Training Website can provide both synchronous instructions, where
students work together in the classroom and an asynchronous, self- study learning
environment where they can learn at their own pace. The link to the UCAT Module was
distributed to the foreign language teachers by sending it by email for teachers to take the
training and to keep the link as a reference in case they need to review some instructions
when needed.
Learning Theories
For the purposes of this training, three primary theories were applied. First is
Behaviorist learning theory in which students acquire new behavior and achieve great
learning outcome based on environmental conditions (Hoy & Miskel,1987). Students learn
best when present in a suitable environment that allows the learners to work using their own
learning strategies, regulate their behavior and get engaged in self-reflection. However, the
external environment and learners own beliefs and values affect this process in a positive or
negative way. Thats why the mission of the trainer is to create a learning environment, urge
and encourage the learner to do better to achieve the goals, applaud students success, and
support failing students in order to get better grades in upcoming tests.
Second is the cognitive theory which focuses on the internal process and connections
that take place during learning. In this theory, we need to focus on the what, how and why
of learning which is the process of learning or gathering facts and information through
hearing, seeing and reading. To have a good learning experience, students should be
exposed to various ways, methodologies, and strategies of teaching. This can easily be done
 12

when using technology in the classroom. For example, Captivate 9 can be used in various
ways to explain how to use UCAT in teaching adults second language, such as audio, video,
and presentations.
Third is constructivist which explains how learners acquire knowledge and learn.
This is the process of expressing ideas, solving problems, and producing information based
on learner experience and back ground knowledge. For example, Captivate 9 provides a
platform for students learning through UCAT to do the following: solve problems, answer
questions that are based on real life experience.
Principles

The following principles are used in this project to ensure that the website includes
interactive materials and design that engage foreign language teachers in the learning
process.
Coherence Principle

Modality Principle

Segmenting Principle

Redundancy Principle

Contiguity Principle

Multimedia Principle

Strategy and Role of the Trainer

There are some challenges when training foreign language teachers how to use
UCAT in teaching second languages especially when it comes to design activities,
instructions, tasks, and assessments for the training module. Foreign language teachers have
different cultural and educational backgrounds and they have diverse abilities, interests, and
needs. The UCAT Training Website includes teaching strategies that keep learners
motivated, engaged and interested in the material given. The UCAT Training Website is an
interactive module that includes meaningful tasks that are related to learners life
experience. There are three stages in task-based activity: pre-task, task, and post-task
(Skehan,1996). In the pre-task stage, learners receive from the trainer necessary information
 13

and knowledge to perform the task. Moreover, in the task stage learners receive the
instructions from the trainer to plan, design, create and produce. The learners in the post-
task stage present and discuss their product and get feedback.
The best kind of learning practice involves the learner in expressing, interpreting and
negotiating meaning. During production stage, learners experience the forms of the learning
in a context that helps them to understand how to use the forms appropriately. Instructors
encourage learners to recognize their own thinking process that leads to developing
independent learner skills. Students proceed with a learning task, observing trainer
performance continuously, and evaluating their own learning when the tasks are completed.
For students to succeed in the learning process they need to know and value the strategies
that assist in their success (Keller, 1987). Bridging is another teaching strategy that allows
students to build on their previous knowledge and establishes a relationship between the
learners and the curriculum. Contextualization is the strategy of getting to know vague
concepts through direct learning. Modeling is when I model a learning concept by speaking
and demonstrating it slowly and clearly.
Delivery System and Media Decisions

UCAT Training Website demonstrates step by step how to use UCAT and how to
develop and design listening, FLO and Reading activities and lessons on UCAT. There will
be engaging activities, assessments, and assignments for learners to work on and that to
ensure the foreign language teachers understand the process of designing online activities
and lessons employing UCAT. The delivery format for this instructional training is online.
The UCAT training Website will be hosted on WIX, a leading cloud-based development
platform. The lectures will be designed and developed using Captivate 9, a rapid responsive
authoring tool that is used for creating e-learning content. The link to the UCAT Training
Website was distributed to the foreign language teachers.
 14

Figure 2: UCAT Training Website home page

Challenges
The primary challenges that this project faced included:
1- Adobe Captivate: Some problems have been encountered while working on
captivate 9 to develop the modules. Here are some of the issues: The navigation
becomes slow at the end of module 2; The play bar disappears in Module 1 and 2.
2- Move to another school: Department E is moving to another school which means
that new meetings need to be conducted with the new administrative staff and
hopefully the new stakeholders will agree to use this website to train foreign
language teachers in how to use UCAT in teaching the second language.
3- UCAT update: New updates have been occurring to UCAT which needed to be
reflected on this product.

Methods
The steps to finish this project followed ADDIE phases: analysis, design, develop,
implement and evaluate.
Analysis. This the first stage of the project which was done earlier by conducting
learner, workplace, and content analysis to gather information needed to come up
with an appropriate solution for the existing problem.
 15

Design and Develop. UCAT training Website includes five sections. The first
section includes the first module which was designed, developed, and tested. This
module instructs foreign language teachers about UCAT terminology and different
UCAT features. The second module instructs foreign language teachers about how to
create Listening and FLO activities on UCAT. The third module instructs foreign
language teachers about how to create reading activities on UCAT. The fourth
module instructs teachers into how to create a lesson on UCAT.
Implement and Evaluate. Ten foreign language teachers volunteered to test the
UCAT Training Website. At the end of the training, the foreign language teachers
evaluated the website by taking the survey on Google Forms. The results of the
evaluation were taken into consideration to continuously improve the website.

Deliverables
The UCAT Training Module contains four modules and summative assessment:
Module one includes instructions, practice, assignment, and evaluative assessment
that examines the trainees on the topic discussed. The practice and assessment
include questions about UCAT terminology and UCAT different features in the form
of multiple choice questions, true or false and drag and drop. The assignment
requires the learners to create a module on UCAT. Appendix C includes sample
pictures from module 1.
Module two consists of instructions, practice, assignment and evaluative assessment.
The practice and assessment questions are in the form of drag and drop asking
students about how to create listening and FLO activities on UCAT. The assignment
instructs teachers to create level 2+ listening and FLO activities on UCAT that must
be shared on UCAT with the trainer. Appendix D includes sample pictures from
module 2.
Module three consists of instructions, practice, assignment and evaluative
assessment. The practice and assessment questions are in the form of drag and drop
asking students about how to create reading activities on UCAT. The assignment
instructs teachers to create level 2+ reading activities on UCAT that must be shared
on UCAT with the trainer. Appendix E includes sample pictures from module 3.
 16

Module four consists of instructions, practice, assignment and evaluative assessment.

The practice and assessment questions are in the form of drag and drop asking
students about how to create a lesson on UCAT. The assignment instructs teachers to
create level 2+ lesson on UCAT that must be shared on UCAT with the trainer. The
teachers can work on a team to create this lesson. The teachers present their lessons
in the lab for trainer and teachers feedback. Appendix F includes sample pictures
from module 4.
Summative assessment includes a survey that collects teachers feedback about the
training materials and if there is any suggestion to improve this website.

After all the modules were completed, ten foreign language teachers volunteered to take the
training, including pre-test and posttest in which usability test was administered. The results
from the administered test were used to create the final report and the UCAT Training
Website.

Resources

Timeline and Costs

Developing this module took around three weeks to finish designing and producing
the instructional modules. The time needed to work on this project was critical due to the
researcher having a full-time job and family obligations. The best time to work on this
project was during the evenings and weekends which was dedicated to designing and
producing the modules. It is worth mentioning that this project costs nothing since there are
no additional costs required to complete this project. Captivate 9 software was used to
develop the modules which were hosted on Wix, a leading cloud-based development
platform that makes it easy for everyone to create a beautiful, professional web presence, to
create the Technology Training Website (Ji &Fu, 2015). I volunteered my hours to complete
this project since it will benefit many foreign language teachers and students at DLIFLC. I
asked my advisor, coworkers and classmates who work in the same teams as me to test this
website for the purpose of receiving their constructive feedback to improve this product. I
collected the feedback from Dr. Fischer who is my professor and advisor at the MIST
program, DLIFLC team members who are expert in the subject and my MIST team who are
 17

expert in the instructional technology field. Moreover, feedback was studied and analyzed to
edit and justify some of the modules content.

Timeline Report
Task Name Duration Start Finish Resource
Design 30 days 9/5/2017 10/5/2017
Content Design 15 days 9/5/2017 9/20/2017 MIST Student
UCAT Training Website 3 days 9/5/2017 9/8/2017 MIST student, Wix
look
Lesson 1 7 days 9/5/2017 9/12/2017 MIST student,
Captivate 9
Lesson 2 7 days 9/5/2017 9/12/2017 MIST student,
Captivate 9
Lesson 3 7 days 9/5/2017 9/12/2017 MIST student,
Captivate 9
Lesson 4 7 days 9/5/2017 9/12/2017 MIST student,
Captivate 9
Formative evaluation ongoing 10-5-2017 MIST student
Development 30 days 10/5//2017 11/5/2017
Website look 3 days 10/5/2017 10/8/2017 MIST student, WIX
Lesson 1 5 days 10/5/2017 10/10/2017 MIST student,
Captivate 9
Lesson 2 20 days 10/5/2017 10/25/2017 MIST student,
Captivate 9
Lesson 3 20 days 10/5/2017 10/25/2017 MIST student,
Captivate 9
Lesson 4 20 days 10/5/2017 10/25/2017 MIST student,
Captivate 9
Summative assessment 5 days 10/25/2017 10/30/2017 MIST student,
Google form
 18

Implementation 10 days 11/6/2017 11/16/2017

Update content based on 10 days 11/6/2017 11/16/2017 MIST student
Feedback

Evaluation 50 days 9/5/2017 11/25/2017

Pre & post test 10 days 9/5/2017 9/10/2017 MIST student
Usability test 4 days 9/5/2017 9/9/2017 MIST student
Summative Evaluation
Finalize Capstone 17 days 11/15/2017 12/02/2017
Capstone Summary 15 days 11/15/2017 12/02/2017 MIST student
Capstone Presentation 15 days 11/15/2017 12/02/2017 MIST student

Evaluation and Testing

Formative Evaluation
The online training website includes four modules. Each module includes a final
assessment that tests students understanding. The first module assessment includes
questions about UCAT terminology, setting up UCAT profile and different UCAT features.
The second module assessment tests teachers ability to create listening and FLO activities
on UCAT. The third module assessment tests teachers ability to create reading activities on
UCAT. The fourth module assessment tests teachers ability to create a lesson on UCAT.
These assessments will be conducted online.

Usability Testing

The usability test was conducted to make sure that the objectives of UCAT training
Website were met and to assure that the navigation in the website and modules were
functioning as expected. Doctor Donald Fischer, my advisor and professor at California
State University Monterey Bay, and three other foreign language teachers took this training
in which they detected some issues and they came up with recommendations and advice to
improve the training. Table 1 sums these issues.
 19

Table 1: Identifying issues and solutions

Issue Is it corrected? Remarks

Module 1: Navigation issues (learner cannot corrected

pause or repeat audio)

Module 1: add progress line Not corrected Unable to resolve

Module 1: Spelling mistakes corrected

Module 1: Mistakes in practice activities and corrected

final exam

Module 1: titles on each page dont exist corrected

Module 1: outdated screenshots corrected

Module 1: change quiz into module 1 final corrected

exam

Module 1: closed captions font is too small corrected

Module 1: add instructions to take module 2 corrected

Module 1: The Pause and Continue Buttons uncorrected In progress

need work in Module 1. I could not get them to
function

Lesson 2, Module 1 Info page the audio cuts off Uncorrected In progress
in mid-sentence

On the matching for Lesson 2, there are still Uncorrected In progress

font differences

On getting to the Module 1 final exam, I got a Uncorrected In progress

blank page

Explaining how to remove the first choice by Uncorrected In progress

clicking on it
 20

Module 2: Navigation issues (learner cannot Uncorrected In progress

pause or repeat audio)

On the Lesson 3, Module 2 True/False, add Uncorrected In progress

feedback on why True is not correct

The T/F question at the end of Lesson 5 I Uncorrected In progress

should add where the + sign they are to click is
located

In the quiz on Activity 2, It is confusing to Uncorrected In progress

learner how to select words
Final exam for module 2 has many navigation Uncorrected I cannot resolve
issues

Module 3: Not allowing learners to redo Uncorrected In progress

quizzes and tests.

Module 4 practice includes information not Uncorrected In progress

provided in the lecture and other issues

Click on the button and could not get back to Uncorrected In progress
where I was in Module 4.

Module 4: Activity 1, T/F. Rational should Uncorrected In progress

be rationale

Module 4 Home Page: Lesson 8 tab is before Uncorrected In progress

the Lesson 7 Tab.

Spelling mistakes uncorrected In progress

All the certificates have your signature that Uncorrected In progress

overlaps the frame
 21

Pre- and Post-testing

The purpose of conducting pre-test is to test foreign language teachers on their

information about UCAT terminology, UCAT features, development of listening, FLO and
reading activities. These questions look the same as the final test in the UCAT module. The
foreign language teachers took the post-test after the training which is exactly the same as
the pre-test. The purpose of conducting such testing is to check if the learner knowledge and
skills have improved and teachers are more confident on working on UCAT to produce
interactive materials that can be used on their daily teaching. The post-test checks if the
objective of the training is met and fulfilled by foreign language teachers. Appendix B
includes the pre-test and posttest questions.
Results
The teachers answered the ten questions present in the pre-test and post-test. Each
question was worth one point. The grades were uploaded from Google forms and copied to
an excel sheet. There are two sets of grades shown as pre-test and post-test grades as we see
in table 1 which also shows the grades distribution. The average of pre-test grades is 6.6
while the average of post-test grades is 9.3. The difference between pre-test and post-test is
shown in table 2. The post-test grades are much better than the pre-test grades which
indicates that the teachers experience the transfer of learning and they become more
knowledgeable about UCAT.
Pretest Post-test
4 8
8 10
9 10
9 10
4 8
4 8
6 9
7 10
7 10
8 10
Table 2
 22

My hypothesis for this training is that foreign language teachers after taking UCAT
training will score high on the posttest which means that the mean of the posttest would be
greater than the mean of the pretest. However, the null hypothesis was that the grades would
not change from pre-test to post-test and the mean of the pretest and posttest would be
equal. I conducted the paired two sample t-test for dependent samples. The degree of
freedom was 9 using one-tail results for a directional hypothesis. We accept the research
hypothesis because the absolute value of the computed t-statistic, |7.36|, is greater than the t
critical one tail value of 1.83. Therefore, the results support a conclusion that the tutorial
contributed to higher posttest scores indicating an increase in the ability to apply UCAT to
DLI learning situations.

Table 3: t-Test: Paired Two Sample for

Means

Variable 1 Variable 2
Mean 6.6 9.3
Variance 4.044444444 0.9
Observations 10 10
Pearson Correlation 0.943456353
Hypothesized Mean Difference 0
df 9
t Stat -7.363636364
P(T<=t) one-tail 2.13247E-05
t Critical one-tail 1.833112933
P(T<=t) two-tail 4.26494E-05
t Critical two-tail 2.262157163
Summative Evaluation
Knowledge check. There are two ways to test learner knowledge in the UCAT
training website. First, learners need to score 80% or more in every module final
assessment. Second, learners need to design a lesson on UCAT that includes the following
parts: activation of Schemata, reading, listening, and task-based activity. It is worth
mentioning that the learners have the opportunities to practice and interact with the
 23

materials before conducting the assessments which are accompanied by effective immediate
feedback.
Survey. The questionnaire was designed in Google Forms to receive learners
feedback about the training and is embedded in the website. The purpose of this
questionnaire is to invest in the input to continuously improve the training module to fit
learners needs and interests. The questionnaire includes eight questions that ask foreign
language teachers about their opinion about the activities, assessment, and materials present
in this module and ways to improve this product. The feedback from the foreign language
teachers is collected and analyzed for the purpose of improving the UCAT Training
Website.

User Experience Survey Results. Seven out of ten teachers answered the survey
questions. They strongly agreed that the objectives of the training were clearly defined and
the content was organized and easy to follow and the time allotted for the training was
sufficient. Four out of seven agreed that the course has practical value in their lives and the
training experience will be useful in their work. Some teachers recommended that this
training should include modules that instruct how to use the other UCAT activities like
matching and hot spot activities. Others recommended that this training should include
modules that instruct how to design level 1, 1+ and 2 activities and lessons on UCAT.

Conclusion
The results from the evaluative assessments indicated that the foreign language
teachers did benefit from this training. The learners now have the ability to develop
interactive activities and lessons on UCAT. Moreover, the foreign language teachers
expressed that the instructions were easy to follow and clear. There are some issues in the
modules that need to be fixed. However, there are some problems that I was unable to fix in
the short term. UCAT Training Website is a unique product because it is the first website
that is developed at our school level that instructs teachers how to use UCAT step by step.
Many improvements will occur to this website in the future to make sure that it supports
DLIFLC goals and learners objectives and needs. Some teachers face difficulties designing
Level 2+ activities and lessons on UCAT which assist students in getting prepared for the
DLPT. This website looks like an online manual for teachers that guides them into how to
 24

develop interactive UCAT lessons to support the Levantine Dialect curriculum course.
Many UCAT modules will be developed in the future to support DLIFLC mission and will
be embedded in UCAT Training Website. Module 5: creating higher level activities on
UCAT; Module 6: creating games on UCAT; Module 7: developing semester one activities
on UCAT.
 25

References

Berge, Z. L., & Collins, M. P. (Eds.). (1995). Computer mediated communication and the

Online classroom: distance learning. Cresskill: Hampton press.

Biggs, J. B. (2011). Teaching for quality learning at university: What the student does.

McGraw-Hill Education (UK).

Brau Maria, Brooks Rachel ILR 101, Language Testing and assessment unit FBI, 2009

http://www.slideshare.net/jasdhillo/ilr-101

Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven

guidelines for consumers and designers of multimedia learning. John Wiley & Sons.

Harasim, L. M. (1995). Learning networks: A field guide to teaching and learning online.

MIT press.

Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students

learn? Educational psychology review, 16(3), 235-266.

Hoy, W. K., & Miskel, C. G. (1987). Theory research and practice (Vol. 278). Educational

administration, New York: Random House.

Ji, Y., & Fu, X. (2015). WEBSITE CONSTRUCTION FOR SMALL COSMETICS

ENTERPRISES.

Keller, J. M. (1987). Strategies for stimulating the motivation to learn. Performance

Improvement, 26(8), 1-7.

Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem

solving. Instructional science, 26(1-2), 49-63.

Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace (Vol. 12).

San Francisco: Jossey-Bass.

 26

Pea, R. D., & Kurland, D. M. (1984). On the cognitive effects of learning computer

programming. New ideas in psychology, 2(2), 137-168.

Roschelle, J., & Pea, R. (2002). A walk on the WILD side: How wireless handhelds may

change computer-supported collaborative learning. International Journal of

Cognition and Technology, 1(1), 145-168.

Sherry, L. (1996). Issues in distance learning. International journal of educational

telecommunications, 1(4), 337-365.

Skehan, P. (1996). A framework for the implementation of task-based instruction. Applied

linguistics, 17(1), 38-62.

 27

Appendices

Appendix A : Summative Evaluation Questions

Rate the following statements

1- The objectives of the training were clearly defined

Strongly disagree disagree undecided agree strongly agree

2- The content was organized and easy to follow

Strongly disagree disagree undecided agree strongly agree

3- The course and the instructor meet your needs as a learner

Strongly disagree disagree undecided agree strongly agree

4- The training experience will be useful in my work

Strongly disagree disagree undecided agree strongly agree

5- The time allotted for the training was sufficient

Strongly disagree disagree undecided agree strongly agree

6- This course has practical value in your life

Strongly disagree disagree undecided agree strongly agree

7- What improvements would you recommend for this course?
 28

Any additional comments

 29

Appendix B : Pre-test and Post-test Questions

1- UCAT webpage URL is *

a) https:\\ucat.dlfilc.com
b) http:\\ucat.dliflc.edu
c) https:\\ucat.dliflc.edu
d) http:\\ucat.dliflc.net

2-UCAT includes the following activities except *

a) multiple choice
b) short answer
c) matching
d) draw and paint

3-In the info page you can do the following except *

a) add module objectives
b) add module description
c) select ILR level
d) change password

4-In UCAT profile feature you can *

a) change your profile picture
b) add an activity
c) preview the module
d) share the module

5-In UCAT social center you can *

a) change your profile picture
b) delete an activity
c) add module objectives
d) add a comment

6-More than one person can work on one module *

a) True
b) False

7-You can access UCAT from anywhere *

a) True
b) False

8-you can share the module with the students by the way of *
a) zip folder
b) link
 30

c) username and password

d) accessing UCAT website

9-you can add to your module all of the following except *

a) picture
b) video
c) audio
d) shapes

10-You can design a lesson on UCAT *

a) false
b) true
 31

Appendix C: Screen Shots of Module 1

32
 33

Appendix D: Screen Shots of Module 2

34
 35

Appendix E : Screen Shots of Module 3

36
 37

Appendix F: Screen Shots of Module 4

38
 39

Release Form