Demonstrating Rupp's Framework • wizirt

Rupp (2013) created a framework to assist researchers in identifying and handling person-misfit in item response theory (IRT). Rupp’s framework consists of five steps,

statistical detection using local and global fit measures (either parametric or nonparametric),
numerical tabulation, or summarization of the incidence of each type of aberrant response pattern,
graphical exploration such as person response functions (PRFs),
quantitative explanation using additional modeling, and
qualitative explanation.

The wizirt package is designed to facilitate the wider spread use of this framework in Psychometric research. With this end in mind, this vignette walks through the use of wizirt for PFA as desccribed by Rupp (2013).

First, users need to load wizirt.

library(wizirt)

The data used in this example are not publicly available, but a practice data set is available in wizirt.

After loading wizirt, users can run a model with the wizirt function

wizirt_fit <- wizirt::wizirt(data = responses, item_type = "Rasch")

Here I specify my data and I designate the model as a Rasch model. The first step of Rupp’s framework asks for global and local person-fit statistics. At this time, wizirt does not calculate local person-fit, but global fit statisics can be calculated using the irt_person_fit() function.

pfa <- irt_person_fit(wizirt_fit, stats = c("Ht"))

These fit statistics, as well as the person abilities and the response patterns (step 2) can be displayed using the print() function.

print(pfa, patterns = T, item_order = "by_diff") %>% 
  dplyr::mutate(dplyr::across(c(ability, std_err, Ht, Ht_cut), .fns = round, 2)) %>% 
  reactable::reactable()

By specifying the item order as “by_diff” the responses patterns are now ordered by the difficulty of the item.

Rupp’s step 3 asks for person-fit to be explored graphically. The nonparametric person response function is used for this in wizirt.

plot(wizirt_fit, type = "np_prf")

Rupp’s step four instructs researchers to try to explain reasons for person misfit. This is done with the irt_model_pfa() function, which allows for the inclusion of predictors.

mod <- irt_model_pfa(wizirt_fit, pfa, bins = 10, predictors = lapply(data[1:50 ,c("School", "center")], scale) )

mod$icc

##                 Ht        
## ICC_adjusted    0.03016229
## ICC_conditional 0.0295526

summary(mod$models$Ht)

## Cov prior  : ids ~ wishart(df = 3.5, scale = Inf, posterior.scale = cov, common.scale = TRUE)
## Prior dev  : 5.2058
## 
## Linear mixed model fit by REML ['blmerMod']
## Formula: Ht ~ (1 | ids) + .
##    Data: mlm_data
## 
## REML criterion at convergence: 91
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.5613 -0.6764  0.0771  0.6626  2.3296 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  ids      (Intercept) 0.001986 0.04456 
##  Residual             0.063851 0.25269 
## Number of obs: 464, groups:  ids, 50
## 
## Fixed effects:
##               Estimate Std. Error t value
## (Intercept)  0.2139900  0.0391069   5.472
## ids          0.0004013  0.0011214   0.358
## ability      0.1654854  0.1083169   1.528
## bin         -0.0082092  0.0040996  -2.002
## School      -0.0109050  0.0141986  -0.768
## center      -0.0035146  0.0136092  -0.258
## 
## Correlation of Fixed Effects:
##         (Intr) ids    abilty bin    School
## ids     -0.739                            
## ability  0.357 -0.495                     
## bin     -0.589  0.011 -0.003              
## School   0.220 -0.317  0.171  0.014       
## center   0.102 -0.126  0.075 -0.013  0.008

These first four steps should be used in conjunction with a qualitative explanation step, something that cannot be done in wizirt.

Demonstrating Rupp’s Framework

Isaac Pfleger

11/2/2020