Educational Research 101 Session 1 Course Overview Learning Objectives 1. Educational Research 1.1 The What 1.

2 The Why 1.3 The How 1.4 Are you ready? 2. Literature Review 2.1 The What 2.2 The Why 2.3 The How 2.4 Framework 3. Research Ethics 3.1 The What 3.2 The Why 3.3 The How Session 2 Learning Objectives 1. Research Questions from Session 1 2. Data Collection 2.1 Quantitative 2.2 Qualitative 3. Data Analysis Process of Analysis Session 3 Learning Objectives 1. Introduction 2. Structure of Report 2.1 Title 2.2 Abstract 2.3 Introduction 2.4 Literature Review 2.5 Methodology 2.6 Results 2.7 Conclusion 2.8 References 3. Reporting of Quantitative and Qualitative Research

Session 1 Course Overview There are 3 sessions in Educational Research for Teachers 101: Session 1: The BIG Ideas behind Educational Research, Literature Review, and Research Ethics Session 2: The BIG Ideas behind Quantitative and Qualitative Approaches (upon completion of Session 1) Session 3: The BIG Ideas behind Research Articles/Reports and Presentations (upon completion of Session 2) For the purpose of updating training hours in TRAISI, ERT 101 is taken to be a 12 hour course. Upon completion of Session 1, participants can add 6 hours as training hours into TRAISI while for both Sessions 2 and 3, participants can add 3 hours each as training hours into TRAISI. On completion of the 3 sessions, a certificate will be sent to participants. Learning Objectives The course is designed to help you to:

understand the what, why and how of educational research apply the big ideas of educational research in real life contexts understand the what, why and how of literature review apply the big ideas of literature review during the educational research process know the 'what', 'why' and 'how' of research ethics apply research ethics to classroom-based research projects

The facilitators for this session are:

Dr Charles Chew (Master Teacher / Academy of Singapore Teachers) Mrs Cynthia Seto (Master Teacher / Academy of Singapore Teachers) Mdm Yau Li Heong (Master Teacher / Academy of Singapore Teachers) Mr Chia Chek Ngee (Academy Officer / Academy of Singapore Teachers)

Instructional Design & Course Development:

Dr Charles Chew (Master Teacher / Academy of Singapore Teachers) Mrs Cynthia Seto (Master Teacher / Academy of Singapore Teachers) Mdm Yau Li Heong (Master Teacher / Academy of Singapore Teachers) Mr Chia Chek Ngee (Academy Officer / Academy of Singapore Teachers)

1 Educational Research 1.1 The What The What of Educational Research

Educational research is a process through which we collect data and analyse them to help us understand an educational issue and possibly how to resolve it. It consists of a systematic and rigorous inquiry or investigation that enables people to understand the nature of problematic events or phenomena.

Research can be characterised by the following: - a problem to be investigated - a process of inquiry - explanations that enable individuals to understand the nature of the problem
A Vignette on Research Research can be visualised as an extension of the daily activities we are engaged in. Let's consider a simple problem, I was baking a cake and it came out burnt.

'Why was the cake burnt?' * Was the oven set at too high a temperature? * Was the cake left in the oven for too long a time?

We ask questions that will lead us to collect information that help generate tentative explanations or hypotheses. These questions will then provide us with the basis for possible solutions to our questions. However, these explanations will need to be verified by authoritative sources. One authoritative source may advise that to in order to avoid burning the cake, the temperature and the duration of baking must be set at specific levels according to size. So, the next time I bake a cake, I will experiment a range of suitable temperatures and also take note of the duration of baking time depending on the size of the oven and the size of the cake being baked. In this way, the chances of the cake being burnt will be reduced. Think for a moment: How is the above story related to the big ideas of research? (Hint : refer to 1.1 The 'What' of Educational Research).

1.2 The Why The Why of Educational Research There are many reasons why we are engaged in educational research. We have listed 3 reasons for discussion:

Expands our knowledge base Improves our classroom practice Helps us make sense of educational policies

a.

Expands our knowledge base

As educators, we encourage our students to be life-long learners. What about us? Research is one way to increase our knowledge base of strategies within and beyond the context of our classrooms. Moreover, when research is codified (recorded) to inform the professional practice of teachers, it is likely to increase the knowledge base of the teaching fraternity. How can this be so? An increase in the knowledge base takes place when we conduct research: i. ii. iii. on emergent (new) pedagogies; to localise (repeat) a research (from another country or educational setting) in the Singapore context; or to compare the effectiveness of different pedagogies

The beauty of research is that when we codify our research findings, other teachers do not need to repeat the research to know the outcome. They have the privilege of applying these findings and using these effective pedagogies in their classrooms! This is a sure way of contributing to the knowledge base of the Singapore education system.

b.

Improves our classroom practice

This flows naturally from the previous point. Educational researchers understand that we do not need to be alone in the classroom. As we work collaboratively with other educators through building on one anothers research or through collaborative research, we can achieve more by standing on the shoulders of giants. Your experience in the classroom would probably be similar to the teaching experiences of the other educators. If research findings are readily available to all of us, we would become more effective and reflective practitioners. This is especially critical for less experienced teachers, who may be tall in passion (Yes, lets do it), but short in direction (Where do we begin?). c. Helps us make sense of educational policies

How many of us have ever felt tongue-tied when our critical stakeholders (the students parents) asked us the rationale of educational policies and their implementation? When we are informed about the rationale of the educational policies through an understanding of related research areas (e.g. holistic education, inquiry-based approach), we can base our lessons and practice on sound research and pedagogies, while avoiding the pitfalls. Through this, we will add value to the school experience of our students. At the same time, we can address the parents' concerns and reassure them that the educational policy makes sense. 1.3 The How The How of Educational Research Broadly, there are 6 steps in the educational research process (Creswell, 2008). After Step 6, you may return to Step 1 to start a new research cycle (to build on what was previously done). For each step, there are a few questions you need to address, in sequence. 1. 2. 3. 4. 5. 6. Identifying a Research Problem Reviewing Literature Specifying a Purpose for Research Collecting Data Analysing and Interpreting Data Reporting and Evaluating Research

1. Identify Identifying a Research Problem a. Specifying a problem: What is the specific problem / focus area you are going to study? Justifying it: Why is this problem important? Suggesting the need to study it: Who will benefit from your research report?

b. c.

2. Review Reviewing the Literature a. Locating resources: Where can you possibly access the books or journals to help you better understand the research problem? Selecting resources: Which of these books or journals that you have read are most relevant in enhancing your understanding of the research problem? Summarising resources: Write a summary of the overall picture you have obtained from the identified books or journals, concerning the research problem.

b.

c.

3. Specify Specifying a Purpose for Research a. Identifying the purpose statement: What do you hope to achieve through the research? Narrowing down the purpose statement into research questions or hypotheses:
o o o o

b.

What exactly do you want to find out? How would you know if your (quantitative: numerical) hypotheses (guesses) are correct? What is the size of the sample you need for your study? What is the sampling process you propose?

4. Collect Collecting Data

a.

Selecting individuals to study: Who are the relevant participants who could provide the answers to your research questions or hypotheses? Obtaining permission: How can you go about getting the participants (or their parents) consent in order to collect information about them? Research ethics guidelines require us to get parental consent for participants who are minors before they participate in the study. Gathering information: How can you collect the most accurate data that you need within the limited time and resources?

b.

c.

5. Analyse and Interpret Analysing and Interpreting Data a. Breaking down the data: What do the collected data really tell you about the research questions or hypotheses? Representing the data: After having analysed the data, how would you interpret the data in relation to your research questions? Explaining the data: What is the best way to summarise your data? Some common ways include tables, graphical representations etc.

b.

c.

6. Report and Evaluate Reporting and Evaluating Research a. Structuring the report: Which report structure would make it easy for the audience to follow your research process? Writing the report sensitively: What are the discriminatory terms you should avoid using in your report? Deciding on audience: How would you distribute your research report so that the critical stakeholders can access it and make use of it? It is best to seek permission from school leaders before sharing your research findings.

b.

c.

1.4 Are you ready? Before you proceed to Literature Review, study the Educational Research Process Cycle below.

Example of the Research Process Cycle (in action) 1. Identifying a Research Problem My students consistently failed to allocate sufficient time for revision (that is, they study at the last minute). This has led to their poor results, or at most, inconsistent good performance. a. Specifying a problem: what is the specific problem / focus area you are going to study? I think it is important to find out how to help them prioritise their time, so as to put a heavier weight on the necessary revision. b. Justifying it: why is this problem important? If we can make sure they place more emphasis on the revision required, they may just do better for their tests and examinations.

c. Suggesting the need to study it: who will benefit from your research report? Obviously, educators (teachers), parents and the students will benefit from this research. I raised a request for funding for the research, and my Principal got the funding from the cluster. Reviewing the Literature a. Locating resources: Where can you possibly access the books or journals to help you better understand the research problem? I started exploring concepts about learning and motivation, through reading books and journals at the NIE library. I went on to access the EBSCOhost database to find full papers on cognitive development. b. Selecting resources: Which of these books or journals that you have read are most relevant in enhancing your understanding of the research problem? I have finally decided to limit my research to concepts on metacognition, as I felt that the students could lack the required metacognitive monitoring skills. c. Summarising resources: Write a summary of the overall picture you have obtained from the identified books or journals, concerning the research problem. Based on 10 most recent research journal articles, I realised that students who performed badly as they allocated insufficient time for revision could be suffering from a judgement of learning error. I summarised some of the commonly proposed metacognitive monitoring strategies and the methods which other researchers have proposed teaching these strategies. Specifying a Purpose for Research a. Identifying the purpose statement: What do you hope to achieve through the research? I hope that this research can provide a simple (and proven) guide on how the students could be trained to self-monitor their learning and revision effectively. b. Narrowing down the purpose statement into research questions or hypotheses: o o o o What exactly do you want to find out? How would you know if your (quantitative: numerical) hypotheses (guesses) are correct? What is the size of your sample? How is the sampling done?

I want to find out which of the metacognitive monitoring strategies would be most effective for my students. I would be able to tell the effectiveness by triangulation their next set of school results versus their previous ones and their self-report on their use of the strategies. I understand that I need to have 65 students per experimental group to support some form of generalisability. This is about 2 classes of students. To have a comparison group, I need to have a total of 130 students (4 classes).

I could not plan for random sampling in a school setting. Therefore, I relied on convenient sampling of using existing classes. This reduced my research generalisability. I worked out some lesson plans with another colleague who also taught 2 classes. Between us, there are 4 classes. My colleague teaches Mathematics, and I teach English. We would randomly assign 1 of each of our classes to form the comparison group while the other remaining class for each subject would form the experimental group. Collecting Data a. Selecting individuals to study: Who are the relevant participants who could provide the answers to your research questions or hypotheses? In our case, we decided to capture data from all the students in our classes, as we felt that their personal characteristics would be spread out among the 4 classes. b. Obtaining permission: How can you go about getting the participants (or their parents) consent in order to collect information about them? Research ethics guidelines require us to get parental consent for participants who are minors before they participate in the study. We got our school General Office staff to help mail and collate parental consent forms. One week after posting the forms, we called those who had not given their replies to explain the benefit of this research. We only based our final analysis on those students whose parents gave their consent. c. Gathering information: How can you collect the most accurate data that you need within the limited time and resources? We arranged for metacognitive training classes after school, and timed the intervention to take place shortly before the common tests. Neutral content classes were given to the comparison group. Selfreported surveys were collected from the students involved, soon after the tests, and triangulation were done. The data collection was repeated for the examination period. Analysing and Interpreting Data a. Breaking down the data: What do the collected data really tell you about the research questions or hypotheses? The collected data showed that about 60% of the students in the experimental group were actively using metacognitive monitoring strategies, and there was a general improvement in the results of these students. There were 2 particular strategies that were most frequently mentioned in the surveys. b. Representing the data: After you have broken down the data, how could you best link your data to your research findings for easy understanding? Are you going to present your data through descriptions, tables, figures or pictures ? I decided to present the student result improvements via a table, so that the readers could see at one glance how the results of those who used the metacognitive monitoring strategies compared to those who did not, and how they also compared to the comparison group in general. I would also be presenting the choice of the strategies via a chart, arranged from the most frequently mentioned to the least. I also quoted the students descriptive feedback about the metacognitive training classes, so that we could reflect on future improvements. c. Explaining the data: What is the best way to help your audience understand how your findings affect them?

10

Drawing links to previous research, I would try to explain how my current findings supported or refuted the proposals of other researchers. I would also elaborate on how to implement the metacognitive training programme, and the pitfalls to watch out for. I could also include some simple questions the parents could ask their children that could begin their own metacognitive monitoring.

A Vignette on Research to Improve the Learning of Mathematics Miss Tan, a mathematics teacher, was concerned that her primary school pupils lacked the motivation to learn from the mistakes they made in their homework. They tended to rely on her to help them with their corrections. She found that a number of her pupils were doing poorly in class tests. She had come across an article about the use of giving written comments to enhance learning. She hypothesized that by writing comments instead of just putting a cross on pupils' homework, it would encourage her pupils to close the learning gaps themselves. She also wanted to know if giving comments would help pupils in their thinking processes. She made a note of two groups of pupils who were doing equally badly in mathematics. For one of these groups, she repeatedly gave them written comments as feedback and taught them to use the feedback to close their learning gaps. After about ten weeks, she compared the scores of the two groups of pupils. She found that the group that made use of her written comments did significantly better. From her interviews with these pupils, she learned that they had in the process, sharpened their meta-cognitive skills and boosted their confidence. Upon reflection, Miss Tan felt that she had learnt much about providing scaffolds and asking questions to provoke deeper thinking. She decided to use the same feedback strategy for the other group. Subsequently, Miss Tan incorporated this approach in her marking and also shared her practice with fellow educators. Activity 1.2 Choose one of these challenge questions: 1) As a teacher-researcher, how did Miss Tan go about understanding her students and their learning? Can you identify the steps she took? 2) How about writing a vignette on one of your professional classroom practices that lends itself to possible research?

2 Literature Review A literature review is a summary of books, journal articles and documents that provides us with past and present information about the topic of the proposed study. Now that you have an idea of the Educational Research Process Cycle, it is important to have a good understanding of how to do a good literature review before you embark on your research. There may be an existing body of knowledge which is relevant and can contribute towards your area of study. How do you tap on these rich resources to inform your research?

11

Lets now gather Three Truly Important Points (TIPs) on Literature Review.

2.1 The 'What' TIP 1 : The 'What' of Literature Review A literature review is a summary of books, journal articles and documents that provides us with past and present information about the topic of the proposed study. Literature review is a bridge between your research idea and the body of knowledge encompassing concepts and theories which already exists in a broader field. (1) In the context of research, each journal article does the following: (a) introduces the research topic (b) (c) (d) (e) (f) (g) identifies work that has already been done on this same topic shows the researchers understanding of the topic provides a rationale for the current research establishes the research context generates research aims and questions demonstrates the researchers critical awareness

(h) shows the researchers ability to organise complex ideas in a logical framework (i) draws out areas where further research is needed

(2) Sources of literature (books, journals, articles in periodicals or newspapers, written reports and online materials) can be broadly classified into: General references:

These references will provide you with information to help you locate other sources. Most of them are indexes or abstracts such as Educational Resources Information Centre (ERIC). ERIC web page http://www.eric.ed.gov/ (subscription for access is needed).

Under ERIC, two very useful preliminary sources are published, namely, Resources in Education (RIE) and

12

RIE provides abstracts of papers presented at education conferences, progress reports of ongoing research studies while CIJE indexes the education journals and journals in related fields.

Primary sources:

These are materials/publications written by authors who were direct observers or participants in the work described. Primary sources in education include journal publications. Secondary sources:

These are materials/publications written by authors who were not direct observers or participants in the events described. Secondary sources in education include text books, research reviews, and educational encyclopedias.

2.2 The 'Why' TIP 2 : The 'Why' of Literature Review The objectives (1) Distinguishing what has already been done from what needs to be done: Usually, researchers do not repeat what has been done, unless they are trying to localise a foreign research project for their particular context. Understanding the structure of the subject: This requires understanding the major theories proposed in previous research. Relating ideas and theory to applications: This requires understanding the work done to see how concepts and theories can be applied in the educational context. Identifying relationships between ideas and practice This requires differentiating between proven ideas (in particular or generalised contexts), and suggested ideas (which could work based on past research). Synthesising and gaining a new perspective: This requires coming up with a useful model for the research study. Discovering important variables relevant to the topic: This is a quantitative approach, in which measurable and useful variables are used in the proposed research model.

(2)

(3)

(4)

(5)

(6)

13

(7)

Rationalising the significance of a problem: This requires justifying how the research would contribute to the knowledge pool of the community. Establishing the context of a topic or a problem: This sets realistic limits as to what can be achieved within the context of the research, and the extent to which the findings can be generalised to other contexts.

(8)

2.3 The 'How' TIP 3 : The 'How' of Literature Review Step 1: Step 2: Step 3: Define the problem as precisely as possible. Formulate the search terms (key words, phrases or descriptors). Select the appropriate general references.

Step 4: Search the general references, such as http://www.lii.org (Librarian Index to the Internet), for the relevant secondary sources. Step 5: Obtain and read the relevant secondary sources.

Step 6: Search the general references for the primary sources, based on the relevant secondary sources. Step 7: Step 8: Obtain and read the relevant primary sources Note and summarise key points in the sources

Remember that a literature review is not a list of items you have read separately, but a synthesised text in which every part contributes to the coherence of the full text. 2.4 Framework The Literature Review Framework The purposeful selection of all pieces of relevant literature can be organised into 3 main blocks. The 3 main blocks in a literature review framework are: the conceptual framework to provide the theoretical base for the topic the contextual review to provide the contexts for the study (local, national or international) the empirical review to provide information on research previously done on the topic

14

3 Research Ethics Classroom Research Ethical Considerations Before embarking on any data collection in research, there are some research ethical considerations for you to take note of. We need to balance the benefits between the researchers and the participants. In the same vein as understanding the big ideas of Educational Research and Literature Review, we will begin the journey of learning the what, why and how of research ethics. 3.1 The 'What' What are Research Ethical Considerations? Some key ideas include:

Research ethics refers to the moral principles guiding research from its inception through to its completion and publication of results and beyond for example, the curation of data and physical samples after the research has been published (Economic and Social Research Council, 2005). All educational research should be conducted within an ethic of respect for:

15

(a) People (b) Knowledge (c) Values

Reference Hopkins, D. (2008). A Teachers Guide to Classroom Research. Open University Press. Burton, D.D. & Bartlett, S. (2005). Practitioner Research for Teacher. London: Sage.

3.2 The 'Why' Why are Research Ethical Considerations important? 3.2.1 Some key principles:

Informed consent (a) Do the people I wish to study have full information about the research, including why and how they have been chosen to participate? (b) Is their consent freely given?

Confidentiality and anonymity (a) How will the information be safeguarded, and the identity of people or institutions be protected? (b) How will anonymity be preserved, for people and institutions?

Ownership of data and conclusions (a) After the collection and analysis, who owns the data and the conclusions? (b) How will the research results be reported and disseminated?

Use and misuse of results (a) What obligations do I have to help my findings be used appropriately, and not to be misused?

Honesty and Trust (a) What is my relationship with the people I am studying?

Reciprocity (a) What do participants gain from my research?

16

Intervention and advocacy (a) What do I do if I see harmful, illegal, wrongful behaviour during my research?

Harm and Risk (a) What might this research do to hurt the people involved? (b) Health and safety issues can be involved in some types of research, and in all cases, the principle of non-maleficence (no harm) takes precedence.

Conflict of interest (a) Questions of power and reliance, along with benefit and trust, are involved in some research situations, and need to handle with care.

3.2.2 Ethical considerations in Academic Writing

Acknowledging the work of others (a) Have the contributions of participants been acknowledged? (b) Who will own the success or failure of the research?

Plagiarism to be avoided (a) This refers to fraudulently presenting ideas as your own when they are not.

Bias-free writing (a) No discrimination or oppression towards any group of people should be allowed.

Reference Punch K. F. (2006). Developing Effective Research Proposals. Thousand Oaks. CA: Sage.

3.3 The 'How' How do we apply Research Ethics in Classroom Research? 3.3.1 Clearance from Data Admin Centre in MOEHQ

17

3.3.2 Checklist before you start Step 1: Preparation Stage

What ethical issues may arise from your research? If your research includes children, what ethical issues could arise from the ages of the children involved? Whose interests is the research serving?

Step 2: Before the data collection

Whose consent do you need? Have you fully explained to the participants the reasons for the research, its methods and possible implications? Have you made clear to participants that they have the right to withdraw at any stage of the research? Are participants assured of confidentiality? How have you made sure that the participants have understood the above? Have you used language that they comprehend?

Step 3: During the data collection

Are you true to your promises to participants? Is the research causing participants the minimum disruptions? Are you storing confidential data safely so that they are not accessible by others? Are the data reliable and valid?

Step 4: During the data analysis

Are findings reliable and valid? Have you been reporting parts of your findings periodically to stakeholders?

Step 5: During the report writing and dissemination

Are you writing the truth? Have you ensured that none of the participants can be identified? Have you considered the different audiences and dissemination approaches for your research?

Reference Hopkins, D. (2008). A Teachers Guide to Classroom Research. Maidenhead. McGraw Hill.

18

Session 2 Dear Teachers, Our heartiest congratulations to you for completing Session 1 and welcome to Session 2 of the Educational Research for Teachers 101 (ERT 101). To help you recall the 6 steps of Educational Research Process, please go to educational research process cycle. You are now going to proceed to Step 4 on 'Collecting Data and Analysing and Interpreting Data'. It's an exciting phase as you explore various methods and find the most appropriate and relevant approaches to address and answer your research problem and questions. The purpose of Session 2 is to give you an idea of how to collect, analyse and interpret data. However, it is not within the scope of Session 2 to go into the details of statistical analysis as this will be covered in Educational Research for Teachers 201 (ERT 201). Before you proceed, sit back and take some time to listen to Associate Professor Christine Lee (Head, Curriculum, Teaching and Learning, National Institute of Education) on her views about research. Once again, welcome to Session 2 of ERT 101 and enjoy the journey to becoming a reflective practitioner! Thank you.

Learning Objectives Session 2 focuses on the BIG Ideas of Qualitative and Quantitative approaches in research and it builds upon what you have learnt about educational research in Session 1. The course is designed to help you to:

understand the characteristics of quantitative and qualitative research understand the common methods used in data collection understand the process of data collection, analysis and interpretation of data

Your facilitators for this session are:

Mrs Cynthia Seto (Master Teacher / Academy of Singapore Teachers) Mdm Rani Viyaya (Master Teacher / Academy of Singapore Teachers) Mr Chia Chek Ngee (Academy Officer / Academy of Singapore Teachers)

19

Instructional Design and Course Development:

Mrs Cynthia Seto (Master Teacher / Academy of Singapore Teachers) Mdm Rani Viyaya (Master Teacher / Academy of Singapore Teachers) Mr Chia Chek Ngee (Academy Officer / Academy of Singapore Teachers)

1. Research Questions from Session 1 In Session 1, we learnt that educational research is a systematic inquiry to enable us to understand the nature of a problem or a phenomenon. There are two parts to this inquiry: the role of theory (literature review) and the central role of data. As such, inquiry does not only entail collection of data or analysing the data collected but more importantly, a correct interpretation of data to answer the questions in the problem or to explain the phenomenon. In Session 2, of Educational Research for Teachers 101, we will concentrate on empirical research. The key concept in empirical research is 'observable information about (some aspect of) the world'. The essential idea in empirical research is to use observable data as a way of answering questions, and of developing and testing ideas (Punch, 2000). Before we discuss data collection, let's revisit the research questions that you have formulated in Session 1. From the questions that you have contributed, we can briefly classify them into various research areas such as:

cognitive domain, e.g. improvement in test scores affective domain, e.g. attitude towards the subject pedagogy domain, e.g. use of ICT or pair-work, etc

Assuming that we have crafted these two research questions for our study: 1) What are the effects of ICT-supported instruction on students attitude towards Mathematics, the use of ICT and academic efficacy? 2) How do ICT-based lessons affect students attitude towards the learning of Mathematics? Let's think for a moment....what data should we collect to answer these two research questions? The decision about what data to be collected is determined by the nature of the problem. There is no one 'recipe' on how to proceed with data collection efforts. As such, teacher-researchers must determine what data will contribute to their understanding and resolution of a given problem. The approach taken is usually based on the desire to understand one's practice and to collect data that is

20

appropriate and accessible.

References: Punch, K.F. (2003). Developing Effective Research Proposals. London: Sage.

2. Data Collection There are 2 approaches to research - quantitative research and qualitative research Quantitative Research is a type of educational research in which the researcher

decides what to study asks specific, narrow questions collects quantifiable data from participants analyses these numbers using statistics conducts the inquiry in an unbiased, objective manner

Qualitative Research is a type of educational research in which the researcher

relies on the views of participants asks broad, general questions collects data consisting largely of words (or text) from participants describes and analyses these words for themes conducts the inquiry in a subjective, biased manner

Session 2 deals with research for both quantitative and qualitative studies, and is based on the view that neither approach is better than the other. Both have their strengths and weaknesses, and that they can, and should be combined where appropriate. The understanding is that the methods and data used (quantitative, qualitative or combined methods) should follow from, and fit in with, the question(s) being asked. After identifying the types of data you are going to collect, the next question to ask will be "How do I collect the data?"

Quantitative Approach Data collection is in the form of numbers

Qualitative Approach Data collection is in the form of words / artifacts (audio-visual) Structure tends to be pre-specified (tight) in Structure tends to be unfolding terms of research questions, research (loose) in terms of design and data research questions, research design and data Begins with hypothesis and theories, Ends with hypotheses and grounded deductive process

21

theory, inductive process Uses experimental methods and quantitative measures to test hypothetical generalisations (theory verification) Seeks to understand phenomenon in context-specific settings (theory generation)

Variables can be identified and relationship Variables are complex, inter-woven measured, tend to be uni-dimensional and difficult to measure, tend to be multi-dimensional
Source: The content in this table is summarised from 'Introduction to Social Research: Quantitative & Qualitative Approaches,' by Punch K.F.
Reference Creswell, J. W. (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (3rd ed.). New Jersey: Pearson Education.

2.1 Quantitative Data collection using the quantitative approach focuses on numbers and frequencies rather than on meanings and experiences expressed in words. The quantitative approach provides information which is fairly reliable and easy to analyse statistically. They are also associated with the scientific and experimental approach. Some common methods of data collection and data are:

performance measures questionnaires or surveys checklists databases

In Research Question 1, we want to investigate if there is a change in students' attitude towards Mathematics as a result of ICT-supported instruction. To conduct this research, the teacher researchers may use a questionnaire such as the "Attitude and Efficacy Questionnaire" to find the mean of each item in the questionnaire. The mean of each scale is calculated for responses to the 'Attitude and Efficacy Questionnaire' which is administered before and after the introduction of ICTsupported instruction. The teacher researcher will then draw a conclusion on the effects of ICT-supported instruction by comparing the means of the pre and postintervention. Performance Measures Teachers construct tests to monitor the performance of students, measure the

22

extent to which each student has attained the learning objectives of a lesson. A well-constructed test will provide a clear measure of attainment for each competency or proficiency, measuring each of the different types of knowledge or skills described in the lesson objectives. A record of results provides the means to track a student's progress in each area of the curriculum, and to gain a clear picture of the overall performance of the class. This type of data is therefore useful for monitoring student performance or evaluating the effectiveness of instruction. Much of the data of student performance is recorded in numeric form, indicating the extent to which the student has attained the desired learning outcomes. As scores may be expressed as raw scores (numbers), as a percentage or in terms of a scale, it is a very convenient way to collect quantitative data from performance measures such as Continual Assessments (CAs), Class Tests, Semestral Assessments (SAs) or even Project Tasks. Depending on the research area, we could also collect data from attendance records, referrals for behaviour incidents or other student performance products, such as homework assignments or workbook pages and analyse them in terms of accuracy, speed, number of items completed, time to complete, and type(s) of errors made.
Reference Stringer, E, (2008). Action research in education (pp 79-80). Upper Saddle River, NJ:Merrill/Prentice Hall.

Questionnaires Questionnaires allow the teacher researcher(s) to collect large amounts of data in a relatively short amount of time. The difference between an interview and a questionnaire is that for the questionnarie, the respondent will write out the responses on the form provided. A Likert Scale is usually used in a questionnaire. Respondents are sometimes asked to respond to a series of statements indicating whether they 'Strongly Agree' (SA), 'Agree' (A), 'Undecided' (U), 'Disagree' (D) or 'Strongly Disagree' (SD) with each statement. Each response corresponds with a point value and a score is determined by adding the point values for each statement. For example, the following point values might be assigned for positive responses: SA=5, A=4, U=3, D=2, SD=1. By assigning these point values, the teacher-researcher would be able to infer whether the students felt positively or negatively about the statement. Questionnaire provides teacher researchers with quantitative (numerical) data which are commonly analyzed by descriptive statistics to find the 'average' or 'mean' value. For example, if the average (or mean) response of the 100 respondents to a particular question concerning the usefulness of a training workshop to their work is 4.2 out of a maximum possible value of 5.0

23

on the Likert Scale, an accompanying narrative to describe it would be that "a large majority of the workshop participants found the training to be useful to their work".
Reference Mills, G.E. (2003). Action Research: A Guide for the Teacher Researcher.(2nd ed.), Pearson, New Jersey.

Checklists When the focus of an educational research is on the students' skills (performance) or behaviours, the data collected are usually done through observation using rating scales. Rating scales for observation may take a simple format of having only two response categories such as

Yes or No Observed or Not Observed Present or Absent

A set of items like this is called a checklist. Checklists are used to standardize observation. Databases In today's digital age, the collection of electronic quantitative data is becoming popular. Databases found in online information resources such as 'Schoolpic', 'School Cockpit' and internal school management systems, provide valuable and easily accessible information for teacher researchers. These databases provide a wide variety of quantitative information which are relevant to the performance of students, their capabilities, and their behaviours over the years. They also provide information about the distribution of the classes, and socio-economic status of the students.

Attitude and Efficacy Questionnaire In this 'Attitude and Efficacy Questionnaire', the instrument uses a 5-point Likert scale with response options of 'Almost Never, Seldom, Sometimes, Often, and Almost Always'. Pupils were asked to respond by indicating on the 5-point Likert scale how often each practice actually took place in the class. The average item mean of each scale was calculated for the experimental group and the control group. Please see below for a snapshot of the questionnaire.

24

Source: The Attitude and Efficacy Questionnaire is a validated instrument from the Curtin University of Technology

2.2 Qualitative
Qualitative methods are ways to collect data which are concerned with describing meaning instead of drawing statistical inferences. They provide a more in-depth and rich description of the issue under study. Some examples of qualitative methods are:

student artifacts e.g. journal observations interviews focus groups

Research Question 2: How do ICT-based lessons affect students attitude towards the learning of Mathematics? To investigate how ICT-based lessons have affected students' attitude towards the learning of Mathematics, we need to collect data that provides information about the affective domain. This kind of data is usually reflected through students' articulation of their emotions, recount of what they did during such lessons or observations of students' behaviours, etc. From the journal, the teacher researchers will look for patterns and recurrent themes to understand students' attitude towards ICT-based lesson in Mathematics.

Journals A students writing in a journal can display his knowledge or understanding of a particular concept, his academic and problem solving ability. It also reveals the student's dispositions, beliefs and attitudes towards the subject. Therefore, journal writings shed light on both the cognitive and affective domains.

25

As Anderson, Herr, and Nihlen (1994) point out: "The journal acts as a narrative technique and records events, thoughts, and feelings that have importance for the writer. As a record kept by a student, it can inform the teacher researcher about changing thoughts and new ideas and the progression of learning(p. 153)." Students' journals can provide teachers with a valuable window into the students' world. Likewise, a journal kept by teachers can also be an opportunity to keep a narrative account of their perspectives of what is happening in their classrooms.
References: Anderson, G.I., Herr, K., & Nihlen, A.S. (1994). Studying your own school: An educator's guide to qualitative practitioner research. Thousand Oaks, CA: Corwin.

Observations Observation is a way of collecting information by observing students performance or reaction under natural and unnoticed situation. It enables teacher-researchers to draw inferences about students meanings and perspectives that they would not likely obtain by relying exclusively on interview data. This is particularly true for sieving out tacit understandings and aspects of students perspectives that they are reluctant to state directly in an interview. The strength of this kind of study lies in the observation of natural behaviour in reallife setting; free from the constraints of other research methods such as interview. Observation as a research technique must be systematic and directed by a specific purpose. The written records of teacher observers are often referred to as field notes. Focus Group Another valuable interview technique is the use of focus groups with several individuals who can contribute to your understanding of your area of focus. One way to think of focus groups is as a group interview where you are trying to "collect shared understanding from several individuals as well as get views from specific people" (Creswell, 2002, p.206). Focus group is a useful technique especially, when the interaction between individuals will lead to a shared understanding of the questions being posed by the teacher researchers. Using a structured or semi-structured interview, the teacher researcher can pose questions to the group and encourage all participants to respond. That means, the interview process will be, ask the question, elicit a response, and pass it on to another participant instead of ask the question, accept the response, and ask another question to the same participant. This is to get as much information out of each participant as you possibly can and, in the process, ensure that all participants have an opportunity to respond. For example, in the video above, the interviewer posed a question on which 'tools' to use in order to understand the concept of ratio. He ensured that every pupil had a chance to give his or her response. When a pupil mentioned that he needed two 'tools' instead of one, the interviewer asked the pupils to raise their hands if they also shared the same opinion. This is to collect data on a shared understanding from the
26

pupils. Ideally, the teacher researcher will use an interview to capture the responses from the focus group and then transcribe the discussion. This is to ferret out the nuances of the focus group interview and the shared understandings that emerge.
References: Creswell, J.W. (2002). Educational research: Planning, conducting, and evaluating quantitative research. Upper Saddle River, NJ: Prentice Hall. Mills, G.E. (2003). Action Research: A Guide for the Teacher Researcher.(2nd ed.), Pearson, New Jersey.

and

qualitative

3. Data Analysis In the educational research process, we have arrived at the stage of 'Data Analysis'. This is the phase for us to review what we have learned and to draw conclusions about what we think the data mean. Data analysis is an attempt by the teacher researcher to process and summarize the data that have been collected in a dependable, accurate, reliable, and correct manner. It is the presentation of the findings of the study in a manner that has an air of undeniability (Wolcott, 1994). Alternatively, data interpretation is an attempt by the researcher to make meaning of the data, to answer the question "So what?" In other words, data analysis tries to report the outcomes or findings of the data collected while data interpretation focuses on the implications or meaning of those findings. The data analysis in quantitative research is straight-forward as the data are collected specifically for statistical tests to draw conclusions with. An example of a simple statistical analysis is the use of means. For example, in Research Question 1, the means for the pre and post intervention are then compared to study the causeeffect relationship between variables so as to draw a conclusion on the effect of the intervention. In qualitative research, the data are the means of generating description, understanding or possible explanation for what is observed. As we go through the data, look for patterns that emerge, such as events that keep repeating themselves, key phrases that participants use to describe their feelings, or survey responses that seem to "match" one another. For example, in Research Question 2, conclusion about students' attitude towards ICT-based learning can be inferred from the way they describe their experiences in their journals.

Reference Wolcott, H.F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA:Sage.

27

Process of Analysis Below is a summary to show the difference between analysing quantitative and qualitative data: Stages in the Process of Quantitative Data Qualitative Data Analysis 1. Raw Data Numbers which represent Words or other artifacts such variables or concepts as pictures, sounds, videos, photos which represent ideas 2. Analysis of Data Usually a one-stage procedure Usually a continuous process, which commences once data using logical analysis to sort collection is completed using data into categories (themes, descriptive & / or inferential exemplars, qualities) or statistical tools for analysis patterns / sequences 3. Interpretation of Data Answering the research question Answering the research with the intention of extending question with the possibility of the findings from the sample to a applying the findings to a wider population similar context / issue

Source: The contents in the table are summarized from 'Social Research' by Sarantakos.

References: Mathison, S. (1988). Why triangulate? Educational Researcher, 17(2), 13-17. Sarantako, S. (1993). Social research. Melbourne: MacMillan Education Australia.

28

Session 3 Dear Teachers, A warm welcome to Session 3 of the Educational Research for Teachers 101 (ERT 101) and our heartiest congratulations to you for completing Sessions 1 and 2. You are now in the last lap of our training for ERT 101 course and in the educational research process cycle, you are now going to start on 'Reporting and Evaluating Research'. Some teachers may find the reporting process quite daunting but let us assure you that this last stage can also be the most rewarding and satisfying. And before we start the session, let us share with you this invaluable tip! The trick to making the reporting process an enjoyable one is to write parts of the report as you are doing your educational research project. Do not leave everything to the last stage! Once again, welcome to Session 3 of ERT 101 as you complete the first milestone of being a reflective practitioner! Thank you. Learning Objectives Session 3 focuses on the BIG Ideas behind research articles/reports and presentations. The course is designed to help you to:

understand the structure of a research report understand an overview of writing a research report in a disciplined way, showing coherence, conciseness and clarity apply the relevant research ethics in reporting

Your facilitators for this session are:

Mdm Alice Aw (Principal Master Teacher / Academy of Singapore Teachers) Mdm Kam Yit Han (Academy Officer / Academy of Singapore Teachers)

Instructional Design and Course Development:

Mdm Alice Aw (Principal Master Teacher / Academy of Singapore Teachers) Mrs Cynthia Seto (Master Teacher / Academy of Singapore Teachers) Mdm Kam Yit Han (Academy Officer / Academy of Singapore Teachers)

29

1. Introduction From Session 1, we learnt that educational research refers to an organised and disciplined attempt to understand and seek answers to problems in educational practices through the collection and analysis of data for the purpose of describing, explaining, generalising or predicting findings. Research is therefore a problem solving activity which addresses a problem, answers questions, tests hypotheses, or explains phenomena. "When we speak of research, we speak of a family of methods that share the characteristics of disciplined inquiry" (Shulman, 1997. p.8). When Lee Shulman, a noted educational researcher, wrote these words, he was offering new researchers two ideas. First, research is disciplined, that is, it is structured and not haphazard: "its data, arguments, and reasoning [are] capable of withstanding careful scrutiny by another member of the scientific community" (Shulman, 1997, p.9). Second, Shulman was pointing out that there is no one way to do research. There are many, but they share the characteristics of being disciplined inquiry. The outcomes of inquiry are documented in a research report. Hence, the purpose of a report is to communicate your research to the educational community; it is like building a bridge between your work and a body of knowledge. Through your documentation and sharing of findings, you add to the repertoire of knowledge, that means, you are a knowledge creator! As there is more than one way of doing research, there is also more than one way of writing a report. For this session, we will adopt a basic structure that is commonly used in preparing a research report.

References Shulman, L.S. (1997). Disciplines of inquiry in education: A new overview. Inr.M. Jaeger (Ed.). Complementary methods for research in education (2nd ed., pp. 3-29). Washington, DC: Amercian Educational Research Association.

2. Structure of Report Writing a report involves making strategic decisions about

what materials and content to include, how to sequence the content, and how to achieve a balance between the need for evidence and the need for clarity

By presenting descriptive details as evidence, one convinces the reader but also risks confusing the reader with information overload. On the other hand, presenting an overview maximises clarity but it may fail to persuade the reader due to lack of evidence and lack of subtlety in reporting and analysis. To avoid the danger of having a report that is either thick and murky or thin and trivial, it is important to have a balance between the need for evidence and the need for clarity.
30

A research report contains basically the following parts: 1. 2. 3. 4. 5. 6. 7. 8. Title Abstract Introduction Literature Review Methodology Results of Findings Conclusions or Implications References

2.1) Title A well-constructed title helps readers in locating articles relevant to their interest. Since concepts in the title are typically indexed and searched electronically, it is important that the words are chosen carefully to convey the contents of the article. Using terms likely to be understood both within and beyond ones immediate research community helps to make the work accessible to a broad audience. The title should be brief and accurately describes the research problem. It should ideally have fewer than twelve words. Assuming that you are writing a report on one of these research problems, suggest a title for your report. 1) What are the effects of ICT-supported instruction on students attitude towards Mathematics, the use of ICT and academic efficacy? 2) How do ICT-based lessons affect students attitude towards the learning of Mathematics? 2.2) Abstract The abstract should provide a summary of the article. It should be self-contained, concise, and accurate. Whenever feasible, it should set forth the research question or problem; descriptions of the research sites, objects, or participants; the conceptual orientation of the study; the methods used for collecting sources of evidence or data; procedures used for analysing the evidence; and the main conclusions and implications. The content of the abstract includes:

background or research statement or hypothesis method(s) used in the investigation key findings of the study

Below is an example of an abstract.

31

This paper reports the findings of the effects of ICT-supported instruction on students' attitudes towards Mathematics. In essence, it reports on the use of ICT and its effect on academic efficacy of pupils in a primary school. A series of ICT-supported lessons were conducted based on the constructivist's view of learning. A total of 80 pupils from two mixedability classes participated in this study. By means of a validated questionnaire, teacherdesigned multiple-choice test papers and students' journals, quantitative and qualitative data were collected and analyzed. Implications of the results were discussed in terms of the use of ICT as a pedagogical tool and motivation of pupils in a primary school. 2.3) Introduction The objective of Introduction is to provide readers with some background information about the problem being researched. The number of words for this section is variable. However, the introduction must contain sufficient information to let the readers know what the study is about, and specifically what aspects the study is addressing. Therefore, you need to state briefly the following:

introduce the phenomena you are studying present what you already know about the problem, and state the outstanding issues in the problem.

If it is a quantitative research, the problem statement should imply a question about the relationship between specified variables and the variables must be clearly defined. As such, it is imperative to link the problem to the body of information available in the field (related to Review of the Literature) and establish the importance of and the need for carrying out the research. Sometimes, a short paragraph under Rationale is included after Introduction to articulate why the problem is worth studying and how solving the problem would help your practice as a teacher or enhance your students' learning. 2.4) Lit Review A Literature Review presents what is currently known about the problem under consideration and thus sets the context for the question or hypothesis of the proposed study. A critical review of the literature will not only position your study but also show how the proposed study fits with the existing body of knowledge. You need to give some background to the research problem, summarise what the literature says about the problem, explain the important terms, and provide any theoretical, conceptual or methodological framework that can lead you to examining the problem.

32

2.5) Methodology In Methodology, you would state clearly the following:

how you have collected the data how many data sources you have used in your study the frequency with which data is collected who your participants are

You would also tell the readers what you did with the data that you have collected and how the data was analysed. These need to be stated in detail so that the readers would be able to make decisions about the validity of your claims or findings. The length of this section varies according to the research problem. Your choice of sub-headings will highlight the way you have conducted your research. Here are some examples of sub-headings that you may use for your report:

Methodological framework Research method(s) Instruments Sample selection Data Collection Data Analysis

2.6) Results The interpretation of research findings, the implication and application of the research findings are discussed in this section. This section is likely to be the most important part of the report. The report should not have any raw and unprocessed data, for example students' raw scores. You would need to organise the data or run the necessary statistical tests before presenting the final form of the data to the readers. A sample of how data are organised for discussion of results of findings in a report is attached below.

Figure 1. A comparison of the mean for each scale in the Attitude and Efficacy Questionnaire

33

Scale Attitude towards Mathematics Attitude towards Computer Use Academic Efficacy

Mean Value (Pre) 3.49 3.84 2.70

Mean Value (Post) 3.63 3.85 2.83

Table 1. A comparison of the mean for each scale in the Attitude and Efficacy Questionnaire

From the table and the graph, it is evident that there is an increase of about 4% in the mean values (from 3.49 to 3.63) for the scale on 'Attitude towards Mathematics' and about 5% for 'Academic Efficacy'. This shows that ICT-supported instruction has a positive effect on students' attitude towards Mathematics and academic efficacy. As there is hardly any difference in the mean values (3.84 vs 3.85) for the scale on 'Attitude towards Computer Use', it is likely that ICT-based instruction does not have any effect on students' attitude to the use of computers. While the results show a positive effect on the 'Attitude to Mathematics' and 'Academic Efficacy', it may or may not be statistically significant. For more rigorous reporting in descriptive statistics, the use of standard deviation* to see the spread of the scores is also useful. An example of the use of standard deviation in a pre-post design in an Inquiry-based Learning course is shown below.

*The concept of standard deviation will be covered in a face-to-face session in Educational Research 201 (ER 201).

34

2.7) Conclusion The conclusion section forms the capstone of the report. The research question or problem statement is revisited to draw conclusions to the question of whether the intervention measures have worked and if so, how they had worked. It indicates what the research findings really mean in terms of informing classroom practice and perhaps how they impact policy at the school level. If the data shows that the intervention actions had failed to reach the desired outcomes, you can suggest reasons for that. It is valuable information to find out why certain interventions do not achieve their intended outcomes. Suggestions for further systematic inquiry can also be included in this section. 2.8) References In this section, all materials you have read and cited are placed under 'References' in your report. Generally, the APA format is used in referencing. Here is the guideline to use APA 6th referencing style: Author, A. A., & Author, B. B. (year). Title of work. Location: Publisher. Author, A. A. (year). Title of chapter. In A. A. Editor & B. B. Editor (Eds.), Title of book (pp. xxxxxx). Location: Publisher. Author, A. A., Author, B. B., & Author, C. C. (year). Title of article. Title of Journal, vol, xxxxxx. doi: xxxxx For more information on APA style, here are a few useful online resources: http://www.apastyle.org/ http://library.nmu.edu/guides/userguides/style_apa.htm

3. Reporting of Quantitative and Qualitative Research Although the format for reporting the research is generally the same for both quantitative and qualitative research, there are some differences in the focus and framing of how they are reported. The table below shows the differences in the way quantitative and qualitative research are reported.

35

Structures Introduction

Quantitative Research

Qualitative Research

*Literature Review

Methodology

Results

Conclusion

The introduction identifies This section identifies the issue on the problem, provides which the study is focused and background information, frames it within the social, cultural, presents the research and/or physical location of particular question(s), and states the groups of people. The purpose of hypotheses to be tested. the study is defined. The literature review Existing definitions and presents what is known interpretations for the problem about the problem and formed within the academic identifies gaps and literature are described and inconsistencies in the critiqued to reveal their underlying academic literature. assumptions. The methodology section This section presents the presents the research philosophical assumptions of the design and describes the form of inquiry applied in the operationalisation of study. It also presents details of hypotheses, sampling research processes, including procedures, instrumentation, procedures for capturing multiple and procedures for data personal and self-stories describing collection and analysis. participant experiences and bracketing procedures that identify key elements, features, structures and themes. The results section presents The results are classified and the outcomes of the study, reassembled into coherent, detailed revealing and interpreting accounts that present participants' the results of data analysis. experience and interpretation of events. Accounts provide an empathetic understanding of participants' lived experience fo the issue investigated. The conclusion discusses Participants' perspectives are the theoretical and practical relocated in the broader social implications of the study, context by comparing and sometimes also presenting contrasting their accounts with recommendations for existing academic and institutional applying the results of the interpretations of the issue. The study. section also explores the implications of the study for policies, programmes and practices related to the issue.

* For some reports, the Literature Review may be incorporated into the introduction, methodology or conclusion section of the report, depending on how the Literature Review helps to illustrate the rationale or main ideas, choice of methodology and supports the conclusion or any theories formulated.

36