Examining the effect of classifying response patterns on an acid-base test by a Guttman-based person-fit index set

Tsai-Wei Huang Shuh-Wang Tsai National Chiayi University 08/11/2012

Introduction
The first time sixth grade students learn the concept of acid-base balance. Two-tier multiple-choice questions (knowwhat then know-how) are effective to diagnose misconceptions. Persons response patterns prevalently were used to analyze students learning misconceptions.

Purposes
To examine the effect of classifying response patterns on an acid-base test by a Guttmanbased person-fit index set

Questions
Q1 : How do the classified clusters based on the BW indices be verified? Q2 : What characteristics of the classified clusters will reveal? Q3 : What characteristics will change for individuals between the know-what test and know-how test?

Rationales of person-fit index

Guttman model BW family indices

Illustration of Guttman-based Response Patterns (N=10 persons, K=15 Items).

Item # p= q= Q = Nq = ID #6 #3 #9 #4 #1 #5 #2 #8 #7 #1 0 T 12 11 10 9 8 7 6 4 2 1 t .80 .73 .67 .60 .53 .47 .40 .27 .13 .07 s .20 .27 .33 .40 .47 .53 .60 .73 .87 .93 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 1 1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 0 0 0 1 1 1 1 0 0 1 0 0 0 0 1 1 0 1 1 0 0 0 0 0 1 1 0 0 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 14 .80 .20 2 2 .70 .30 3 3 .70 .30 3 9 .60 .40 4 15 .60 .40 4 6 .60 .40 4 7 .50 .50 5 1 .50 .50 5 5 .40 .60 6 13 .40 .60 6 11 .30 .70 7 12 .20 .80 8 8 .20 .80 8 4 .20 .80 8 10 .10 .90 9

BW family indices
Person-facet
W (Carelessness) B (Guessing) C (Capability) M (Misconception)
Wi =
Ti j =1

( 1 uij ) ( q* qij ) iT [( K 1 ) / 2 ]

Bi

j = Ti + 1

u ij ( q i j q * ) iT

[( K 1 ) / 2 ]
Ti

Ci

j =1

u ij ( q * q i j ) iT

[( K 1 ) / 2 ]
K

Mi

j =Ti +1

(1 - uij ) ( qij q* ) iT [( K 1 ) / 2 ]

Method
Subject 67 valid 6th grade students in Taiwan. 41 boys (61.19%) and 26 girls (38.81%).

Instrument
20 two-tier multiple-choice acid-base questions Item difficulty
Easy : .65 <P<1.0 (7 items) Moderate: .4 <P <.64 (7 items) Hard: .0 <P <.39 (6 items)

Item discrimination
High discrimination: D >.50 (9 items ) Middle discrimination: D >.30 (8 items ) Low discrimination: D <.20 (3 items )

Example 1
Know-what question (Answer) ( ) According to the weather report, rain water from most parts of the world is acidic rain. What do you think the nature of acidic rain is? (1) Acidic (2) Neutral (3) Basic (4) Uncertain Know-how question (Reason) ( ) What causes it to possess such property? (1) Because of openings in the ozone layer. (2) Because factories, vehicles and scooters release acidic substances into the air, therefore rain water is acidic. (3) Because factories release basic substances into the air, therefore rain water is basic. (4) Whether the nature of rain water is acidic, neutral or basic, it depends on the substances in the air at the moment.

Example 2
Know-what question (Answer) ( )There are many types of aqueous solutions in our daily lives and one of them has a slippery feeling when touched. What kind of property might it posses? (1) Acidic (2) Basic (3) Neutral Know-how question (Reason) ( ) What do you think the best reason for this is? (1) Soap water feels slippery (2) Tap water feels slippery (3) Peeling pears feel slippery

Scoring system
S1. Dichotomous scoring for 1st tier items as kk1 S2. Dichotomous scoring for 2nd tier items as kk2 S3. Coding cognitive styles truly known: kcogn = 1, if (kk1 = 1 & kk2 = 1) guessing: kcogn = 2, if (kk1 = 1 & kk2 = 0) careless: kcogn = 3, if (kk1 = 0 & kk2 = 1) unknown: kcogn = 4, if (kk1 = 0 & kk2 = 0) S4: Summing the number of each cognitive type across 20 items within subject, and create four variables as TK (truly known), GS (guessing), CR (careless), and (UK) unknown to serve as the predictors of discriminant analysis and multinominal logistic regression procedures on the criteria of group classifications from clustering.

Four cognitive styles of the two tier items

Classification by BW indices

Analysis
BW indices estimation The BW indices (W,B,C,M) were calculated through the first and second tier items (i.e., right or wrong). Changes from the two tiers of test can be compared through standardized indices. Classification: Three cluster analysis methods based on would classify samples into 2-4 groups based on the BW indices by
Between method (Average linkage between groups) Ward method K-mean method

Verification: The four variables (TK, GS, CR, UK) would serve as the predictors on the criteria of group classifications from clustering by two identifying approaches
Discriminant Analysis (DA)~ linearly prediction Multinominal Logistic Regression (MLR) ~ nonlinearly prediction

RESULTS
Classification

Both identifying approaches confirmed the BW-based classifications were valid with very high hit rates of predictions, especially the nonlinear multinominal logistic regression. Since the small number of cases (n=2) clustered by the Between method in the first two types of groups (Group = 2 and 3) and that (n=4) by the K-mean and Ward methods in the final group (Group =4) for both DA and MLR procedures, we will discuss the properties of clusters by the K-mean and Ward methods in the first two types of groups.

Properties of 2 clusters
.PHDQ
:DUG

 * UR X S 

0 & %
* UR X S

0 & %





















Properties of 3 clusters
.PHDQ
:DUG

 0 * UR X S  & %

 0 * UR X S  & %

:          
         

& '% ( %& $ & F C F G F DEC D G ' $ ( DB EC A A D F BA BC D F )0 HI

'% $ (

H1 H1

H1 L1

)1 HP  H 6

"! )  #$ @A

76 98

)2

HQ

S 4" 3 4 Y b cd d Y a Y b a cd ba c ef !5 " 30 ! R 97 8T 9S 7 RI

 

 

L1 H1
 eg VU YXW ` eU   eh

H1 A3

p XV Vi iWq Xp

Change between know-what and know-how

fW 

 

  

Conclusion & Discussion

1. Two and three groups of classification by the methods of K-mean and Ward were satisfactory. 2. Two-tier test (answer/reason) design confirmed the BW-based classifications were valid with very high hit rates of predictions, especially predicted by the nonlinear multinominal logistic regression. This implies the BW indices can effectively predict students practical cognitive styles of acid-base learning.

3. Properties of clusters by the methods of K-mean and Ward were similar;

High level cognitive students with low misconceptions tend to be less guessing and careless than the low cognition students do in 2 g - roup classification. Low level cognitive students tend to guess; middle level ones tend to slip.

4. Examining the changes of standardized personfit indices between the know-what test and the know-how test can provide much diagnostic information.

Limitations and Suggestions

The BW indices can effectively predict students practical cognitive styles of acid-base learning, but it still need more essential evidence. Although a two-tier test is not easy designed (in this study, the discriminations were not satisfactory), it still possesses rich diagnostic information. Future studies might apply this device to other subjects for more diagnosing students learning misconceptions, especially combined with the WB person-fit indices.

Thanks for your listening!