NEW YORK INSTITUTE OF TECHNOLOGY

SCHOOL of EDUCATION
Master of Science in Instructional Technology

Assessment & Evaluation

EDIT 690

Case Study Assignment

Afra Alzaabi & Rasha Alzaabi
EDIT 690 Case Study Assignment

Introduction

The impulse of integrating technology in today’s classrooms became universal and this requires a
good establish for introducing more effective professional development programs. Based on that,
we went through three stages with specific group of teachers. Firstly, we have undertaken the needs
analysis since we wanted to determine the needs of using instructional technology for the teachers
at Shaikh Zayed Center for Qur’an memorization. Utilizing surveys, interviews, and field-notes
gave us a good indicator to the current participants’ needs. The gathered data showed that
participants need to learn ways of saving and sharing files, and submitting documents, so we
decided to introduce Google Drive and Google Classroom. After ensuring the needs of
participants, we proposed an instructional effectiveness plan for evaluating the results because it
is very crucial to offer relevant training that meets participants’ needs. The next stage which came
after was the implementation stage. After meeting with the senior manager and showing her the
data tools results (participants’ needs), we agreed on implementing the PD.

The main objectives for the PD were using Google Drive to save and share files and using Google
Classroom to submit documents. There are two types of assessment to use with participants:
formative and summative assessment. To ensure that participants have met the learning outcomes
of the addressed goals, we used different tools to help us. From our experience, it is very
fundamental to see evidence of learners’ understanding and how they use their learning in new
situations and this occurs through using assessment tools. According to the Understanding by
Design framework, the second stage of backward design ensures that objectives are properly
assessed through using assessment tools (McTighe & Wiggins, 2012).

To start with summative assessment, adult learners are self-directed learners where they can use
their critical thinking and connect between their experience and the new knowledge (Gutierrez,
2015). An online survey created using Google Forms was distributed between participants to
evaluate the learning process at the last day. Moreover, to evaluate participants’ learning at the end
of PD sessions, we asked them to create folders (project-based) in Google Drive to save specific
files shared through email and share them with their colleagues. After looking at participants’
work, we were able to know which instructions would be adjusted. For the formative assessment,
we focused on two tools; questioning strategies, and observations. To have successful sessions,

Afra Alzaabi & Rasha Alzaabi 2

EDIT 690 Case Study Assignment

questions were asked from instructors and participants which helped us to receive immediate
feedback on the learning process. While one of us was introducing a new idea, the other one was
walking around the room to make sure participants were on the right track and to write down notes.

Problem Statement

After analyzing the data that we got from the assessment tools, we determined that our participants’
desire changed in some instructional technology areas, such as saving and sharing documents
because the survey showed that 85.5% of the participants agreed that they want to learn about it
They were using USB drives which is not effective as it spread viruses and cause documents lose.
Therefore, we recommended the professional development course about using Google Drive in
order to help them to overcome the conditions.

Saving documents was one of the major issues that teachers confront. Based on the assessment
tools; one of the interviewed participants stated “As a result of having two duty-shits, the huge
number of documents make me frustrated”. Moreover, another teacher commented on the survey
that “losing files from computer” was the main instructional technology issue that she used to have.
Thus, it has been recommended to have a PD course with 3 sessions to teach them how to create a
google drive account, create folders and save documents in those folders.

Sharing documents was another issue as most of the teachers agreed. One of the participants
emphasized in the interview that exchanging the teaching materials, such as power point and word
documents by using USB devices is not practical at all especially with this big number of teachers
at the center, and another one commented on the survey that “sharing teaching materials with other
teachers” was the main instructional technology issue that she used to have. Therefore, it has been
recommended to have a PD course with 1 session to help the teachers to be able to make sharable
links for the desired documents and send it among each other by using the Google Drive app.

Moreover, as the observation/field-notes showed that submitting frequent documents, such as

students’ monthly reports and attendance sheet to the center’s administration was another issue
that teachers have; they used the same ‘traditional’ method of putting them in a USB drive or even
printing them out to be submitted. So, it has been recommended that participants continue the PD
with two more sessions to learn about another Google app; which was the Google Classroom. An

Afra Alzaabi & Rasha Alzaabi 3

EDIT 690 Case Study Assignment

account has been created for the center and all the teachers has been invited to that classroom to
submit their documents easily.

Literature Review
Formative and summative assessments are two types of educational evaluations with different
purposes. These evaluation techniques can be used at various periods across the educational cycle.

Formative assessment is a continuous process which provides coaches with information to improve
training programs while it is happening. Based on our research, criteria and goal setting,
questioning strategies, and observations were used as formative tools. These tools were used
during the four sessions to ensure participants’ awareness of their learning and to collect more
feedback on the learning process for immediate improvement. According to Derrell (2015), a
reliable formative assessment tool is the one that offers both teachers and learners opportunity to
inspect the results and apply them for future practices.

Questioning was one of the most formative tools used by coaches and participants. In fact, we
focused on raising the level of questions because participants are adults. We noticed that when we
raised the level of questions, participants asked more thoughtful questions which raised the level
of understanding. Based on research done on formative assessment in adult learning by (Swain,
Griffiths, and Stone, 2006), it is more effective to use higher level questions with adults to inquire
more thinking rather than recalling information.

The second formative tool used during the sessions was observations. We focused on observing
participants during the training programs since we wanted to identify their learning difficulties
during the learning process. Furthermore, we wrote down some notes, so we used them to compare
their understanding level through all sessions. One of the advantages of using observations was
that we introduced one more session to ensure participants’ understanding in using Google
Classroom as they faced some difficulties in uploading files. Observing learners provides valuable
and immediate information about participants’ progression and what strategies can be used to
improve their learning as emphasized by Regier (2012).

Summative assessment is an effective method that coaches use in instructional technology PD

courses to measure the end results. In our research, we used two tools to focus on the outcomes of

Afra Alzaabi & Rasha Alzaabi 4

EDIT 690 Case Study Assignment

the training; an evaluation online survey and a given task where the participants created Google
accounts and signed in to the Google Drive to save and share folders with other teachers, as well
as a Google Classroom account to upload documents, they were supposed to do it by themselves
individually to examine their performance. Although the course has been done in different weeks
(one session per week), we were able to do the summative assessment to measure the effectiveness
of the whole process even after a period of time of having the first session. Based on the research
of Schilling & Applegate (2012) about the best method for evaluating educational impact, they
emphasized that “Summative evaluation represents a point in time usually either immediately after
training or longitudinally”.

In adults’ learning situation, the survey method is an authentic summative assessment tool that
gives the participants a sense of being part of the development process which could show valid
results as Wiggins (1998) highlighted from the Indiana University research. Moreover, it is very
effective for learners to express themselves in a manner that highlights valid aspects about their
learning to help the instructors measure the effectiveness of their teaching in order to make future
changes to enhance the performance.

The second summative assessment tool (the task that participants should fulfil) is another example
of using suitable method for adult learners because their learning is goal-oriented and the task was
direct to their goal of the instructional technology area of need which is saving and sharing
documents. Therefore, the results of the summative assessment of this method was direct and clear.
We found this so relevant to Pullagurla (2014) when he stated that “Once the learning goals have
been identified, it is now imminent to align the learning activities such that these objectives are
fulfilled within a certain period of time”.

Case Study

1. Description of Assessments used

Both summative and formative assessments showed results that helped us as coaches to determine
what went well and what needs changes in the PD training. For example, the observation in the
formative assessment indicated that few participants needed more time to finish the required task
whereas the majority were able to finish the task on-time. Therefore, we implemented an
immediate intervention and paired the participants to work collaboratively and help each other.

Afra Alzaabi & Rasha Alzaabi 5

EDIT 690 Case Study Assignment

The questioning method was effective to a certain point as some of the participants were able to
interact with the high level ones while the rest of them could respond to the basic questions only.

We think that the immediate intervention of action that we did during the PD based on the
formative assessment affected the summative assessments results positively. The survey showed
a very high level of satisfaction about the course as the participants wrote complements as well as
suggestions for the next courses. Furthermore, the participants responded very well to the task-
based tool at the end of the training which indicated a clear success for meeting the objectives of
the course.

2. Data Generated by Assessments

Utilizing results of assessments is an important stage because it assists us as coaches to evaluate

the teaching content of training programs. We came up with some questions like, have the
participants met the learning outcomes? Are the assessment tools relevant to the learning
outcomes? Do they effectively reflect what is being asked of participants? In fact, the data analysis
gave us an answer to these questions. Qualitative and quantitative are two different types of data
which was required for the results analysis.

Quantitative Data

Quantitative data refers to collecting information through numbers and it can be presented through
graphs or tables. In this PD, we created an online survey using Google Forms application as a
quantitative data source. The purpose of the survey was to collect more data about participants’
understanding of the course content. To clarify, it showed that most of participants are able to save
and share files in Google Drive whereas some of them are struggling using Google Classroom.
Using assessment tools that showed qualitative data was a good choice because we were able to
determine what the next step to take with participants is and it gave us an overall picture of their
current level of understanding as highlighted by Surbhi (2016). This tool was not hard

Qualitative Data

On the other hand, qualitative data is the type of information that cannot be measured. We focused
on observations as a qualitative data tool to obtain more reliable information. Our participants are
adults, so they were more self-conscious when they were observed. This tool was not easy to

Afra Alzaabi & Rasha Alzaabi 6

EDIT 690 Case Study Assignment

implement because we had to select which areas to focus with participants during the session as
well as writing down some notes. Although observing participants was not easy, but it gave us an
insight into the bigger picture and it helped us to develop the rest of the training as mentioned in
the University of Surrey research (n.d).

Interpreting Data

Combining qualitative and quantitative data tells us the effectiveness of applying the training
program and it also refers to the effectiveness of assessment tasks. In our case, we insisted on using
different types of assessment tools because we wanted to get more accurate evaluations.
Furthermore, putting interpretations and data together showed that we were able to meet the target
objectives through using various assessment tools. Additionally, using various types of data raised
the level of validity as we were able to depend on the data to evaluate participants’ understanding.

Decisions about Teaching and Learning based on Assessments

After reviewing the results of the evaluation tools we decided that the Google Drive will be
implemented officially in the center for the teachers to start using it after getting the permission
from the administration and then they will start using the Google Classroom, and the reason we
put them in this sequence is based on the responses of the online survey (summative assessment
tool) after the PD course. Most teachers agreed that Google Drive is most important for them and
they want to start using it immediately because their priority is to keep their files safe and protected
as well as share them with each other without the risk of losing them. Consequently, Google
Classroom will be used after the teachers become more familiar with the first google app and after
the center creates an official account to be circulated to all teachers and used to submit all the
required documents.

In addition, the assessments did not only lead to current discussions, but they also informed other
indicators for future plans, such as using the Google apps from mobile devices and not being
limited to the use of PC or desktop computers. Moreover, several teachers highlighted in the post-
training survey that would be so helpful to use those apps with the learners and parents (of the KG
section) as well as to enhance the teaching and learning and collaborate by using the instructional
technology.

Afra Alzaabi & Rasha Alzaabi 7

EDIT 690 Case Study Assignment

Self-reflection of Collaborative Research

Collaboration in a research has so many advantages for both; researchers and the target group. In
our case, we found it very effective to collaborate in this project because we constructed our
knowledge about evaluation research by sharing and discussing the concepts to come up with
innovative ideas. Belcher (2010) highlighted that “When you have different people collaborating
on a project, then you get a greater sense of creative input”. We really found it useful to discuss a
subject and see it from different points of view to expand the perspective of our thinking toward
the desired objective.

On the other hand, collaborative research could be challenging in many areas, such as time
consumed. The time spent in negotiating the subject sometimes may cause a delay on the research
process. As SivaKumar (2016) emphasized, collaborative work can take much longer than
anticipated. In addition, using similar language and following a logical sequence was another
challenge we faced during writing the report. However, we concluded that collaboration was a
great method that aimed at gaining powerful results throughout our success in planning, having
open communication, respecting opinions and appreciating the effort of the other side.

Conclusions & Recommendations

To sum up, the main purpose of making professional development courses is to make changes in
specific and determined areas that participants need in order to enhance their performance. As
coaches of 21 st century; the ‘instructional technology’ is a major target to focus on and to be
harnessed for a very wide variety of fields, such as schools, hospitals, companies, etc. In our case,
teachers of Shaikh Zayed Center was the focus group because we believe that it is very essential
for educators of this era to use technology to adapt their teaching and learning. As stated in the
Hanover Research (2014) “Professional development provides educators the opportunity to
understand new advancements and adapt their teaching styles and pedagogy to make effective use
of available educational enhancements”.

After going through the process of evaluation research; starting from conducting the needs
assessments tools moving to data gathering then analysis, we determined the instructional
technology area of need that our participants desired. Consequently, we made the PD course

Afra Alzaabi & Rasha Alzaabi 8

EDIT 690 Case Study Assignment

recommendation to train the teachers on how to use two tools of Google apps; the Google Drive
and the Google Classroom in order to meet the main objectives of this intervention which are
sharing, saving and submitting documents.

As coaches, we need to use evaluation tools to measure the effectiveness of the training course in
order to recognize the pros and cons of the process and to know what changes should be done in
the future to get better results. In our research, we used an online survey, as a summative
assessment tool to get feedback from the participants about the PD and to know their opinion as
well as their recommendations for next courses. We also used another summative assessment tool
where teachers had to complete a task individually to assess their ability and to determine whether
they met the objectives of the PD training or not. Furthermore, we used two formative assessment
instruments; the questioning strategies and the observation method to monitor the learning process
and to provide ongoing feedback. According to Carnegie Mellon University (2016), in adults
learning setting; the feedback of formative assessments can be used by coaches to improve their
teaching and by learners to improve their learning.

To ensure continues improvement of participants’ performance, coaches should always make

structured interventions to meet the variable needs. In instructional technology setting, the
interventions cannot be ended at any point because the technology is developing constantly.

Afra Alzaabi & Rasha Alzaabi 9

EDIT 690 Case Study Assignment

References
Applegate, R & Schilling, K. (2012). Best methods for evaluating educational impact: a
comparison of the efficacy of commonly used measures of library instruction. Retrieved on
(January 17, 2018) from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3484955/

Belcher, L. (2010). Advantages & Disadvantages of Collaboration in the Workplace. Retrieved

on (January 20, 2018) from: http://smallbusiness.chron.com/advantages-disadvantages-
collaboration-workplace-20965.html

Carnegie Mellon University. (2016). What is the difference between formative and summative
assessment. Retrieved on (January 21, 2018) from:
https://www.cmu.edu/teaching/assessment/basics/formative-summative.html

Derrell, T. (2015). Formative vs. Summative Assessment: What's the Difference. Retrieved on
(January 20, 2018) from: https://www.aiuniv.edu/blog/2015/june/formative-vs-summative

Gutierrez, K. (2015). 4 Elements to Effective Adult Learning. Retrieved on (January 20, 2018)
from: https://www.shiftelearning.com/blog/effective-adult-learning-neuroscience

Hanover Research. (2014). Professional Development for Technology Integration. Retrieved on

(January 18, 2018) from:
https://ts.madison.k12.wi.us/files/techsvc/Professional%20Development%20for%20Technology
%20Integration.pdf

Indiana University. (n.d). Summative and Formative Assessment. Retrieved on (January 19,
2018) from: https://citl.indiana.edu/teaching-resources/assessing-student-learning/summative-
formative/

National Research and Development Center. (n.d). Research Briefing. Retrieved on (January 20,
2018) from: http://dera.ioe.ac.uk/22504/1/doc_4335.pdf

Pullagurla, A. (2014). 6 Top Facts About Adult Learning Theory. Retrieved on (January 20,
2018) from: https://elearningindustry.com/6-top-facts-about-adult-learning-theory-every-
educator-should-know

Afra Alzaabi & Rasha Alzaabi 10

EDIT 690 Case Study Assignment

Regier, M. (2012). 60 Formative Assessment Strategies. Retrieved on (January 20, 2018) from:
http://www.stma.k12.mn.us/documents/DW/Q_Comp/FormativeAssessStrategies.pdf

SivaKumar, S. (2016). Collaborations: Pros and Cons. Retrieved on (January 21, 2018) from:
http://www.ascb.org/compass/compass-points/41032-2/

Surbhi, S. (2016). Difference Between Qualitative and Quantitative Data. Retrieved on (January
18, 2018) from: https://keydifferences.com/difference-between-qualitative-and-quantitative-
data.html

University of Surrey. (n.d). Introduction to Research. Retrieved on (January 19, 2018) from:
http://libweb.surrey.ac.uk/library/skills/Introduction%20to%20Research%20and%20Managing%
20Information%20Leicester/index.htm

Wiggins, G & Mctighe, J. (2012). Understanding By Design Framework. Retrieved on (January

21, 2018) from:
http://www.ascd.org/ASCD/pdf/siteASCD/publications/UbD_WhitePaper0312.pdf

Yale University. (2016). Formative and Summative Assessments. Retrieved on (January 19,
2018) from: https://ctl.yale.edu/Formative-Summative-Assessments

Afra Alzaabi & Rasha Alzaabi 11