Escolar Documentos
Profissional Documentos
Cultura Documentos
Chapter 7: Measurement Chapter 8: External Selection I Chapter 9: External Selection II Chapter 10: Internal Selection
McGraw-Hill/Irwin
Employment:
Chapter Outline
Importance and Use of Measures Key Concepts Measurement Scores Correlation Between Scores Quality of Measures Reliability of Measures Validity of Measures Validation of Measures in Staffing Validity Generalization Staffing Metrics and Benchmarks
Collection of Assessment Data Testing Procedures Acquisition of Tests and Test Manuals Professional Standards Legal Issues Disparate Impact Statistics Standardization and Validation
7-4
Assume you gave a general ability test, measuring both verbal and computational skills, to a group of applicants for a specific job. Also assume that because of severe hiring pressures, you hired all of the applicants, regardless of their test scores.
How would you investigate the criterion-related validity of the test? How would you go about investigating the content validity of the test?
What information does a selection decision maker need to collect in making staffing decisions? What are the ways in which this information can be collected?
7-5
Key Concepts
Measurement
the
process of assigning numbers to objects to represent quantities of an attribute of the objects amount of the attribute being assessed
Scores
the
Correlation
a
between scores
Examples
Tests of applicant KSAOs Job performance ratings of employees Applicants ratings of their preferences for various types of job rewards
7-7
of measurement process
(a) Choose an attribute of interest (b) Develop operational definition of attribute (c) Construct a measure of attribute as operationally defined (d) Use measure to actually gauge attribute
Results
of measurement process
Scores become indicators of attribute Initial attribute and its operational definition are transformed into a numerical expression of attribute
7-8
Measurement: Definition
Process
7-9
7-10
Measurement: Standardization
Involves
Controlling influence of extraneous factors on scores generated by a measure and Ensuring scores obtained reflect the attribute measured
Properties
of a standardized measure
Content is identical for all objects measured Administration of measure is identical for all objects Rules for assigning numbers are clearly specified and agreed on in advance
7-11
Measurement: Levels
Nominal A given attribute is categorized and numbers are assigned to categories No order or level implied among categories Ordinal Objects are rank-ordered according to how much of attribute they possess Represents relative differences among objects
Interval Objects are rank-ordered Differences between adjacent points on measurement scale are equal in terms of attribute Ratio Similar to interval scales equal differences between scale points for attribute being measured Have a logical or absolute zero point
7-12
measures
used to assign numbers to attribute are predetermined, communicated, and applied through a system
Subjective
Scoring
measures
system is more elusive, often involving a rater who assigns the numbers
Research
results
7-13
Scores
Definition
Measures provide scores to represent amount of attribute being assessed Scores are the numerical indicator of attribute
Central
Exh. 7.2: Central Tendency and Variability: Summary Statistics Percentage of people scoring below an individual in a distribution of scores
Percentiles
Standard
scores
7-14
Discussion questions
Imagine
and describe a staffing system for a job in which there are no measures used. Describe how you might go about determining scores for applicants responses to (a) interview questions, (b) letters of recommendation, and (c) questions about previous work experience.
7-15
Scatter diagrams
Used to plot the joint distribution of the two sets of scores Exh. 7.3: Scatter Diagrams and Corresponding Correlations Value of r summarizes both
Correlation coefficient
Values can range from r = -1.0 to r = 1.0 Interpretation - Correlation between two variables does not imply causation between them Exh. 7.4: Calculation of Product-Movement Correlation Coefficient
7-16
7-17
7-18
7-19
significance
Refers to size of correlation coefficient The greater the degree of common variation between two variables, the more one variable can be used to understand another variable
Statistical
significance
Refers to likelihood a correlation exists in a population, based on knowledge of the actual value of r in a sample from that population Significance level is expressed as p < value
Interpretation -- If p < .05, there are fewer than 5 chances in 100 of concluding there is a relationship in the population when, in fact, there is not
7-20
Quality of Measures
Reliability Validity Validity Validity
of measures
Consistency of measurement of an
attribute
A measure is reliable to the extent it provides a consistent set of scores to represent an attribute
Reliability
of measurement is of concern
Both within a single time period and between time periods For both objective and subjective measures
Exh.
7-23
error
score = true score + error Deficiency error: Occurs when there is failure to measure some aspect of attribute assessed Contamination error: Represents occurrence of unwanted or undesirable influence on the measure and on individuals being measured
7-24
7-25
Coefficient alpha
Should be least .80 for a measure to have an acceptable degree of reliability Minimum level of interrater agreement - 75% or higher
Interrater agreement
Test-Retest reliability
Concerned with stability of measurement Level of r should range between r = .50 to r = .90
Intrarater agreement
For short time intervals between measures, a fairly high relationship is expected - r = .80 or 90%
7-26
of reliability
Relationship to validity
Reliability of a measure places an upper limit on the possible validity of a measure A highly reliable measure is not necessarily valid Reliability does not guarantee validity - it only makes it possible
7-27
Degree to which a measure truly measures the attribute it is intended to measure Accuracy of measurement
Exh.
Accuracy
Exh.
of prediction
7-28
7-29
Discussion questions
Give
examples of when you would want the following for a written job knowledge test
a
7-30
7-31
7-32
Predictors must be accurate representations of KSAOs to be measured Predictors must be accurate in predicting job success
Validity
7-33
7-34
7-35
7-36
Content Validation
Content
validation involves
Demonstrating the questions/problems (predictor scores) are a representative sample of the kinds of situations occurring on the job
Criterion
A judgment is made about the probable correlation between predictors and criterion measures
Used
in two situations
When there are too few people to form a sample for criterion-related validation When criterion measures are not available
Exh.
Validity Generalization
Degree
Situation-specific
generalization
7-38
Discussion questions
Assume you gave a general ability test, measuring both verbal and computational skills, to a group of applicants for a specific job. Also assume that because of severe hiring pressures, you hired all of the applicants, regardless of their test scores.
How would you investigate the criterion-related validity of the test? How would you go about investigating the content validity of the test?
What information does a selection decision maker need to collect in making staffing decisions? What are the ways in which this information can be collected?
7-39
quantifiable measures that demonstrate the effectiveness (or ineffectiveness) of a particular practice or procedure
Staffing
metrics
metrics
as a means of developing
7-40
procedures
standards
7-41
Legal Issues
Applicant flow statistics Applicant stock statistics Lack of consistency in treatment of applicants is a major factor contributing to discrimination
Standardization
Example: Gathering different types of background information from protected vs. non-protected groups Example: Different evaluations of information for protected vs. non-protected groups
Validation
If adverse impact exists, a company must either eliminate it or justify it exists for job-related reasons (validity evidence)
7-42
Ethical Issues
Issue
Do
individuals making staffing decisions have an ethical responsibility to know measurement issues? Why or why not?
Issue
Is
it unethical for an employer to use a selection measure that has high empirical validity but lacks content validity? Explain.
7-43