- A. The role of measurement theory in personality research

- 1. as a tool for prediction and control

- 1. Attributes as Latent Variables

- a) physical and psychological constructs are unobservable
- b) inferred from observed effects

- a) temperature and heat
- b) weight and mass
- c) test score and ability

- a) scales defined in terms of their sensitivity to transformations
- (1) ordinal transformations
- (a) preserve order but not distance: x>y <==> x' > y'
- (b) without zero
- (c) with zero

- (2) partial orders
- (a) some distances are preserved
- (b) some distances are indeterminant

- (3) interval scales preserve distance
- (a) x-y > s-t <==> x'-y' > s' - t'
- (b) Kelvin <-> Celcius <-> Farenheit

- (4) ratio scales preserve distance from 0
- (a) x/y > s/t <==> x'/y'>s'/t'
- (b) interval with zero point
- (c) Meters to Miles

- (1) ordinal transformations
- b) the meaning of transformations
- (1) statistics based upon observed data
- (2) inferences about latent variables
- (a) what kind of inferences can one make about latent variables?
- (b) main effect differences
- (c) interactions

- 1. Measures of Central Tendency

- a) Mode as the most frequent observation
- b) Median as the middle measure
- (1) is not sensitive to distributional shape
- (2) not sensitive to transformations

- c) Mean -- multiple types
- (1) Arithmetic
- (2) Harmonic -- reciprocal of arithmetic mean of reciprocals
- (3) Geometric -- the nth root of n products

- a) range
- b) interquartile range
- c) variance
- (1) variance of composites

- (1) variance of composites

- a) covariance
- b) regression as the best fitting linear relationship
- (1) expressed in scale units

- c) correlation
- (1) simple correlation
- (a) geometric mean of regression slopes
- (b) scale free

- (2) types of simple correlations -- different forms of the PPMCC
- (a) Pearson product moment correlation
- (b) Spearman rank order
- (c) Point-Biserial
- (d) Phi

- (3) multiple correlation -- n predictors, 1 criterion
- (4) partial correlation -- removing the effect of other predictors

- (1) simple correlation

- a) estimates of rank order
- (1) classic test theory
- (a) parallel tests
- i) alternate form
- ii) stability

- (b) congeneric measurement
- (c) correction for attenuation
- (d) estimation of true score

- (a) parallel tests
- (2) consistency
- (a) domain sampling
- i) item-domain correlations
- ii) test - test correlations
- (1) coefficient alpha
- (2) coefficient beta

- (b) alpha as average of all possible split halfs

- (a) domain sampling
- (3) generalizability theory

- (1) classic test theory
- b) item response theory
- (1) estimates of attribute
- (2) estimates of item difficulty
- (3) 1, 2 and 3 parameter models

- a) internal and external sources of validity
- b) face ("faith") validity
- c) concurrent validity
- d) predictive validity
- (1) the use of tests in decision theory
- (a) selection ratio
- (b) success rate

- (2) utilities
- (a) valid positives and valid negatives
- (b) false positives and false negatives

- (1) the use of tests in decision theory
- e) construct validity
- (1) convergent
- (2) discriminant
- (3) incremental

- f) Threats to validity
- (1) Domain specificity versus Generality
- (a) reliability is maximized if items are completely redundant
- (b) predictive validity is maximized if items are completely independent

- (2) Response styles
- (a) "Yea saying"
- (b) Social Desirability
- (c) Extreme response set

- (1) Domain specificity versus Generality

- a) reliability+validity=causal model
- b) importance of alternative models
- c) goodness of fit

- 1. Methods of Keying

- a) Rational Keying
- (1) Ask items with direct content relevance
- (2) Example: California Personality Inventory
- (3) Problems:
- (a) Not all items predict in face valid direction
- (b) Need evidence for validity

- b) Theoretical Keying
- (1) Ask items with theoretical relevance
- (2) Example: Jackson Personality Research Form
- (3) Problems:
- (a) Theoretical circularity
- (b) Need evidence for validity

- c) Empirical Keying
- (1) Ask items that discriminate known groups
- (a) Administer wide range of items to People in General and the the criterion group
- (b) Select those items that discriminate criterion group from People in General
- (c) Select items that are most independent of each other
- i) reduces redundancy
- ii) reduces effect of any single domain of items

- (d) Create scale made up of discriminating items
- (e) Validate on different group
- i) Validation on hold out sample
- ii) Cross validation to determine shrinkage

- (2) Example : MMPI, Strong-Campbell
- (3) Problems
- (a) What is meaning of scale?
- (b) Need to continually develop new scales for new groups

- (1) Ask items that discriminate known groups
- d) Homogeneous Keying
- (1) Select items to represent single domains
- (2) Example: Cattell's 16PF, Eysenck's EPI, EPQ
- (3) Problems:
- (a) Garbage in-Garbage out
- (b) need evidence for validity

- (4) Methods
- (a) Factor Analysis
- i) Factor analysis model: R = FF' + U2
- (1) Explains the inter-test covariances
- (2) explains the reliable (common) part of test variance

- ii) Vocabulary
- (1) Hyperspace and hyperplanes
- (2) Communalities versus uniqueness
- (a) amount of test variance explained by all factors
- (b) "row wise"

- (3) Eigenvalues and eigenvectors
- (a) amount of total variance explained by one factor
- (b) ³Column wise"

- (4) Residual Matrix = R - FF'
- (5) Methods of extraction
- (a) centroid
- (b) principal factors
- (c) minimal residual
- (d) maximum likelihood

- (6) Simple Structure
- (7) Rotations and Transformations
- (a) Orthogonal rotations
- i) VARIMAX
- ii) QUARTIMAX
- iii) BiFactor

- (b) Oblique transformations
- i) OBLIMIN
- ii) BiQuartiMin

- (a) Orthogonal rotations
- (8) Higher order - 2nd strata factors
- (9) Goodness of fit of model
- (a) size of residual correlations

- (10) Factors versus components
- (a) variables are sums of (hypothetical) factors
- (b) components are sums of (observed) variables

- (11) Indeterminancy of factor scores
- (12) parameters versus observables
- (13) clusters as group factors

- iii) Exploratory
- (1) Number of factors problem
- (a) Scree test
- (b) Chi square
- (c) Very Simple Structure
- (d) Parallel analysis
- (e) Eigen values > 1

- (2) Goodness of fit
- (3) parsimony versus fit -- multidimensional preference function

- (1) Number of factors problem
- iv) Confirmatory
- (1) theory testing
- (2) goodness of fit of one model versus another
- (3) meaning of failure to fit
- (a) non-normal distributions
- (b) large sample sizes ==> non-fit

- v) Problems:
- (1) Sensitive to number of subjects versus number of items
- (2) not encouraged for items (low communalities)

- i) Factor analysis model: R = FF' + U2
- (b) Principal Components Analysis
- i) model: R = CC'
- ii) Components can be described at data level
- iii) Components are sums of items
- (1) Describe the data as they are
- (2) describe covariances as well as variances

- iv) Items are sums of factors
- (1) factors go beyond the data to estimate common part
- (2) factors are latent

- v) for more than ‰ 20 variables, these distinctions become less important (pragmatically)

- (c) Cluster Analysis "Poor man's factor analysis"
- i) Non-hierarchical
- (1) find most similar pair
- (2) combine them
- (3) add items to this pair until alpha of total fails to increase
- (4) repeat a-c on remaining items

- ii) Hierarchical
- (1) find most similar pair
- (2) combine them
- (3) repeat a-b until some criterion is reached

- iii) Advantages
- (1) Simple to understand
- (2) Aims for direct solution (clusters as group factors)
- (3) Robust to poor correlations

- iv) Disadvantages
- (1) not well defined as maximizing any particular criterion
- (2) "cut and try"

- i) Non-hierarchical
- (d) General problems -- when not to factor analyze:
- i) too many factors in the data
- ii) too few factors in the data
- iii) unique factor -- defined on only 1 test
- iv) variables are complex
- (1) use Factorially Homogeneous Item Dimensions (FHIDs)
- (2) Homogeneous Item Composites (HICs)

- v) artificially inflated or deflated correlations
- (1) item overlap inflates correlations
- (2) ipsative scoring deflates correlations

- vi) heterogeneity of population
- (1) within group versus between group corrations
- (2) correlations of aggregates ‚ aggregate correlation

- vii) homogeneity of population
- (1) restriction of range will reduce correlations

- viii) bad distributions
- ix) different difficulty levels of scales or items
- (1) factorially homogeneous test varying in difficulty will produce multiple factors
- (2) Guttman simplex pattern

- x) Small sample sizes
- (1) error of correlation depends upon sample size
- (2) do not need 10* number of variables

- xi) low correlations and low communalities (e.g. items) make structure harder to identify if using factorial techniques

- (a) Factor Analysis

- e) Does it make a difference?
- (1) For theory construction
- (2) For predicting criteria

- a) Review theory of attribute to be measured
- (1) Convergent measures
- (2) Discriminant measures

- b) Write items based upon theory
- (1) items drawn from different facets of theory
- (2) items balanced for response styles
- (3) screen items for readability, bias, understandability
- (4) Include "hyperplane stuff"
- (a) possible related constructs
- (b) theoretically important alternatives

- c) Define target population
- (1) Consider issues of homogeneity/heterogeneity
- (2) Consider issues of generalizability

- d) Administer items and record responses
- (1) Monitor for serious, engaged test taking
- (2) Double check for data entry errors

- e) Examine the distribution and search for outliers
- (1) data entry errors
- (2) uncooperative subjects

- f) Form proximity (correlation) matrix
- g) Extract optimal number of factors or clusters
- (1) statistically (chi square and maximum likelihood)
- (2) psychometrically (maximize alpha, beta, VSS)
- (3) for interpretation (to maximize understanding)

- h) Form scales based upon these factors, clusters
- (1) score salient items
- (2) drop non salients

- i) Purify scales -- item analysis
- (1) high correlation with scale
- (2) low correlations with other scales
- (3) low correlations with measures of response styles
- (4) moderate levels of endorsement

- j) Validate against other measures of same and different constructs
- (1) Assess reliabilty
- (a) internal consistency
- (b) stability

- (2) Demonstrate convergent, discriminant and incremental validity

- (1) Assess reliabilty

- 1. Self Report

- a) Direct subjective
- (1) empirical scales: MMPI
- (2) factorial scales: EPI/16PF
- (3) rational scales: PRF

- b) Indirect/projective
- (1) TAT
- (2) Rorschach

- c) Indirect/objective
- (1) Cattell objective test battery

- d) Indirect/other
- (1) Kelly Construct Repetory Grid
- (2) Carroll INDSCAL

- e) Structured interviews

- a) Peer ratings
- b) supervisory ratings
- c) subordinate ratings

- a) unobtrusive measures
- b) historical record
- (1) GPA
- (2) Publications
- (3) Citations

- a) neurometrics
- b) "lie detection"

- a) OSS stress tests
- b) New faculty job talks
- c) Clinical graduate applicant interviews

- 1. dimensions of peer ratings and self report