ESL EFL Writing The Pragmatic Tests Cloze

The Cloze procedure was first applied as a reliability measure with native speakers by Taylor (1953). It has since been demonstrated in many studies to have substantial concurrent validity as an integrative test of overall proficiency, in English as a second language. In the studies carried out by Oller & Conrad (1971), Oller (1972), Stubbs & Tucker (1974), Irvine, Atai & Oller (1974) Hinofotis (1980) and Edith Hannia & May Shikhani (1986) high correlations scores on an established measure of language proficiency. Cloze is considered an integrative rather than a discrete point test because it draws at once on the overall grammatical, semantic and rhetorical knowledge of the language. To reconstruct the textual message, learners have to understand key ideas and perceive inter within a stretch of continuous discourse and they have to produce, rather than simply recognize an appropriate word for each blank. The focus of the task involved is more communicative than formal in nature and it is therefore considered to reflect a persons ability to function in the language. According to Moller (1981), both cloze procedure and paragraph writing are integrative in nature. They require the learner to draw upon several language skills simultaneously and involve complex processing of language while the focus is on content. Both require the production of language rather than mere recognition of correct items, although writing may be considered to include the communicative dimension more directly. There is some disagreement among researchers about the suitability of cloze to test high order language abilities. According to Alderson (1979) cloze provides a measure of core linguistic skills of a relatively low order. According to a number of other researchers like, Chihara, Oller, Weaver and Chavez (1977), Bachman (1982) and Brown (1983), cloze procedures can test not only lower order linguistic skills, but also higher level ability involving discourse constraints across sentences. Types of Cloze Some cloze use rational deletion procedure to ensure the inclusion of cohesive items. Bachman (1982) in his study used the rational deletion procedure. Some other studies use systematic deletion procedure where the blanks occur at regular intervals. In a study conducted by the American University of Beirut cloze passages were carefully selected so that blanks systematically include both syntactic and cohesive factors and therefore covered higher order skills. Edith Hanania and May Shikani (1986) in their article in TESOL

Quarterly (Vol.20) report a recent research carried out at the American University of Beirut where the inter relationship of three types of measures A standardize ESL test, a cloze and a written composition test was studied. The results indicate that there is a high correlation between cloze and written composition test. Oller (1979) gives detailed instruction for the selection of cloze passage and for scoring procedure. Selecting the material The difficulty level of the passage should be suited to the level of the learners. The passage should be of some general topic which does not require technical knowledge. The standard test construction technique known as the fixed ratio method involves deleting every n the word (where n usually varies from five to ten) and replacing each one with a standardized blank (usually about fifteen typed spaces). The standard length of cloze is fifty items thus the passage length is approximately fifty times n. Another procedure is to delete words on some variable ratio usually decided by a rational selection procedure e.g. delete only content words. Taylor (1957) demonstrated that the every nth word deletion technique yielded cloze items of somewhat greater reliability and validity than deliberate selection of words to be deleted, i.e. deleting only function or content words. Scoring criteria There are two methods of scoring a cloze passage. The exact word method The word in the original passage is considered as the correct one and any other word which the testee might supply is not accepted. Contextual appropriateness Any word supplied by the testee which is appropriate to the context is accepted. General guidelines given by Oller for contextual appropriateness are as follows: If it is the exact word as in the original passage it is marked correct. If not, it should be checked to see if it fits the immediate surrounding context; whether or not it violates any local constraints in the same sentence or surrounding phrases. It should also be checked to see if it is consistent with all of the preceding and subsequent text (this includes previous and subsequent responses in other blanks as filled in by the examinee). If the response passes all these checks it is marked correct, otherwise it is considered incorrect.

Naccarate and Gilmore (1976) have shown that contextually appropriate scoring can be done either by a single rater working alone or by several raters, with equivalent results. Having discussed in detail the pragmatic tests, it is time to compare them with discrete point and integrative tests.

Comparison - Discrete point / Integrative / Pragmatic Tests John Carroll (1961) first proposed the distinction between discrete point and integrative language tests. Though these two are not always different for practical purposes there are two basic differences. ? ? The theoretical bases of the two approaches contrast markedly. Predictions concerning the effects and relative validity of the two procedures differ.

Discrete tests take language skills apart and attempt to test the knowledge of language one bit at a time. Integrative tests put the skills back together and attempt to assess learners capacity to use many bits all at the same time and possibly while exercising several presumed components of a grammatical system and perhaps more than one of the skills or aspects of skills. Pragmatic tests go a little beyond the integrative tests. The pragmatic tests meet the pragmatic naturalness criteria while the integrative tests do not reach up to that point. So unlike the integrative tests, pragmatic tests invoke and challenge the developing grammatical system of the learner. This requires processing sequences of elements in the target language subject to temporal contextual constraints. In addition to this, pragmatic tests are such that learners need to relate the linguistic sequences to extralinguistic contexts in meaningful ways. Implications $ The above discussions indicate that pragmatic tests like dictation and cloze can be used effectively to test language proficiency. These two tests are valid and reliable measures of language use. They are practical because it is not very difficult to prepare the material, administer the test, score the test and interpret the results. They also have diagnostic value, and error analysis based on the results could offer suggestions for further instructional programmes. $ Scoring of cloze passages could be based on contextual appropriateness and that of dictation could be of the correct word-in-sequence method.

(See 2.3.6 biii) $ Since objective, multiple-choice items do have some advantages, some vocabulary and grammar items could be tested using this method. $ To test the communicative efficiency, a letter could be given for which they are to write the reply. The reply could also be given in part thereby structuring the learners response partly so as to make the evaluation more effective. This could again be scored using both contextual appropriateness criteria and correct word-in-sequence method. ESL EFL Writing Tests The Pragmatic Test Types

Tests which meet the pragmatic naturalness criteria There are two closely interrelated criteria of construct validity, which need to be imposed on language tests to find out if they meet the pragmatic naturalness criteria. The test must cause the learner to processes either produce or comprehend or possibly to comprehend, store and recall, or some other combination temporal sequences of elements in the language that conform to normal contextual constraints which may be linguistic constraints or extra linguistic constraints. The test must require the learner to understand the pragmatic interrelationship of linguistic contexts and extralinguistic contexts. Some tests which meet the pragmatic naturalness criteria are as follows: a. b. Dictation Cloze

c. Combined Cloze and Dictation A cloze passage is prepared deleting single words, parts of words or sequences of words and given to the learner in the written form. The same passage, without any deletions, is heard by the learner either live or on tape. The learner is asked to complete the cloze passage with the help of what he hears. d. Oral cloze procedure A cloze passage with deletions is dictated for the learners to copy down and complete the deleted portions with appropriate words. e. Dictation with interfering noise Taped verbal material is super imposed with `white noise`. If the linguistic context under the noise is fully meaningful and subject to the normal extralinguistic constraints, this procedure qualifies as a pragmatic testing technique. For

example, if it is a telephone conversation with one party calling from a market place, the noise superimposed will have sounds of the market place. f. Paraphrase recognition Learners are asked to read a sentence, paragraph or longer passage and then select from several alternatives the one, which best represent the central me aning or idea of the given piece. g. Question answering Four answers are given for each question. Learners listen to the question orally and select the best answer. There may be a short dialogue followed by a question. Four or five answers are given and learners select the best answer. This type of test is used in TOEFL examinations. h. Bilingual syntax measure A picture or cartoon is shown and learners are asked some questions based on the picture. Burt, Dulay and Hernandez (1975) made learners answer questions based on colourful cartoon style pictures. i. Oral interview Where learners are asked questions as would happen in a normal interview for a job or admission to a course. j. k. l. Composition or essay writing Narration Translation Which is similar to the on taking place in a real life context.

Having discussed the nature and types of pragmatic tests in general, it would be useful to consider dictation and cloze in more detail because they meet the criteria like validity, reliability and practicality, being easy to construct, administer and score.