Think in terms of the purpose of tests and the consistency with which the purpose is fulfilled/met

Validity and Reliability

Neither Valid nor Reliable Reliable but not Valid

Fairly Valid but not very Reliable

Valid & Reliable

Validity

Depends on the PURPOSE E.g. a ruler may be a valid measuring device for length, but isnt very valid for measuring volume Measuring what it is supposed to Matter of degree (how valid?) Specific to a particular purpose! Must be inferred from evidence; cannot be directly measured

Reliability

Consistency in the type of result a test yields

Time & space participants

Not perfectly similar result but very close-to being similar When someone says you are a reliable person, what do they really mean? Are you a reliable person?

Types of validity measures

Face validity or Content Validity Construct validity

1. Convergent 2. Discriminant

Criterion validity
1. Predictive 2. Concurrent

Types of Validity
FACE or CONTENT- Simplest form of validity in which researchers determine if the test seems to measure what is intended to measure CONSTRUCT - Summary judgment of whether the questions measure what you intended them to measure - CONVERGENT - Do the questions relate to other questions that measure the same construct - DISCRIMINANT - Do the questions not relate to questions they should not relate to

Types of Validity

CRITERION - evidence involves the correlation between the test and a criterion variable (or variables) taken as representative of the construct. In other words, it compares the test with other measures or outcomes (the criteria) already held to be valid

Types of Validity
- CONCURRENT- refers to the degree to which the
operationalization correlates with other measures of the same construct that are measured at the same time
-

PREDICTIVE - refers to the degree to which the operationalization can predict (or correlate with) other measures of the same construct that are measured at some time in the future

Reliability

Measure of consistency of test results from one administration of the test to the next Generalizability consistency (interwoven concepts) if a test item is reliable, it can be correlated with other items to collectively measure a construct or content mastery A component of validity

Length of assessment

Test retest Give the same test twice to the same group with any time interval between tests Equivalent forms (similar in content, difficulty level, arrangement, type of

assessment, etc.)

Measuring Reliability

Give two alternative forms of the test to the same group in close succession Split-half Test has two equivalent halves. Give test once, score two equivalent halves (odd items vs. even items) Cronbach Alpha (SPSS) Inter-item consistency one test one administration

How to improve Reliability?

Quality of items; concise statements, homogenous words (some sort of uniformity) Adequate sampling of content domain; comprehensiveness of items Longer assessment less distorted by chance factors Developing a scoring plan (esp. for subjective items) Ensure VALIDITY

Measurement Error
Sources of Measurement Error

Xo = XT + XS + XR
The observed score or measurement The true score of the characteristic Systematic error Random error

Measurement Error
Examples of Systematic Error Social Desirability Cross Cultural Differences Personality Differences Seasonality Examples of Random Error Persons Mood Question Ambiguity Fatigue

Thank You !