Multidimensional IRT and DIF in R with mirt

Рет қаралды 9,544

Күн бұрын

☕If you would like to support, consider buying me a coffee ☕: buymeacoffee.com/statsguide8
For one-on-one tutoring/consultation services: guide-tree-statistics-consult...
I offer one-on-one tutoring/consultation services for many topics related statistics/machine learning. You can also email me statsguidetree@gmail.com
For rcode and dataset: gist.github.com/musa5237
Tutorial video of how to run the Multidimensional Item Response Theory (MIRT) and detect Differential Item Functioning (i.e., bias items) for ordinal polytomous response data in R. Packages used are mirt. The model used was the multidimensional extension of the Graded Response Model (GRM).
Confirmatory, exploratory, and bifactor models were reviewed. In addition, DIF with anchor items was reviewed.
Discussion board for questions related to the package where I received help: groups.google.com/g/mirt-package
Dataset was responses to the Generic Conspiracist Beliefs Scale (GCBS; Brotherton, French, and Pickering, 2013). The instrument is designed to measure degree of conspiracy theory beliefs. The goal was to measure the psychometric properties of the instrument and detect items in the instrument that may be more biased toward one gender.

Пікірлер: 34

@statsguidetree 3 жыл бұрын

Here is the rcode (because some symbols are not allowed in the description): ################################################################################ ########## overview # Multidimensional Item Response Theory (MIRT) models # Exploratory/confirmatory MIRT models # Bifactor models # DIF with anchors # Multidimesnional IRT models (i.e., more than one factor) are generalizations # Traditional unidimensional IRT models (e.g., 1PL, 2PL, 3PL, GRM, etc.) # can each be gneralized to estimate additional dimensions (i.e., factors) # These Multidiemsional IRT models can be confirmatory (i.e., number of factors # in the scale are known prior) or exploratory (i.e., number of factors in # the scale are not known prior). ################################################################################ ########## dataset # Generic Conspiracist Beliefs Scale (GCBS; Brotherton, French, # and Pickering, 2013) response dataset. # Source: openpsychometrics.org/_rawdata/ # Load dataset downloaded 15 items with scale containing 5 levels # (1=S.Disagree,2=Disagree, 3=Neutral, 4=Agree, and 5=S.Agree) gcbs

@user-qd3lz1wu2m 4 ай бұрын

Thank you for your great video on DIF with ‘mirt’. I managed to do DIF analysis with your detailed instructions. I have a few questions, though. I’ll be very thankful if you answer them. 1. When we examine DIF, we are basically interested in the item difficulties across the two groups. I don’t follow when the manual says, “Determine whether it's a1 or d parameter causing DIF”. How can the slope parameter cause DIF? Isn’t DIF a property of the intercept? 2. What is the ‘RMSD_DIF’ function? How is it used and interpreted? 3. I’m working with large scale international educational data where around 50 countries take part. I need to examine country DIF. I know that in this context we don’t do pairwise comparisons, but we estimate a pooled international ICC (all countries combined) and then compare the country ICCs with the pooled international ICC. Can you please provide the codes for this? Thanks for your help in advance. Afshin

@namelessone5022 Жыл бұрын

You have no idea how much you helped me with mirt. I used to play with other irt packages but this one is much more complex than others and often I get error messages that I don't when using other packages. Great thanks to you again, and for including bifactor model example (which I'm studying too). Rewatching your video, I noticed something. You added more d parameters than I typically found in other examples using mirt. You have d1-d4, which I assume is because of your dependent var having 5 categories? On this topic too, I would like to confirm one thing. Because in one psychological paper, I have seen the author testing for DIF using only which.par('c=a1') but not using 'd'. It wasn't explained so I wonder what's the difference? I believe 'a1' tests for only non-uniform DIF, while 'd' tests for uniform DIF, and 'a1'+'d' tests for both types of DIFs. If that's the case, would it be best to test for both types of DIFs by having 'a1'+'d'? I generally see no reason why you would test for just one type of DIF? what's your take on this?

@andrej.mentel Жыл бұрын

Thank you for a lecture. I would like to ask some additional questions: 1. Model fit - do you recommend to use M2 statistics and global fit indices such as CFI, TLI, RMSEA etc. as a measure of model appropriateness? Or for a model comparison? 2. What measures do you recommend for the decision whether to use a bifactor or a two-dimensional model? 3. How can you in the bfactor function specify the other polytomous IRT models? For example, generalized partial credit model - does it allow the argument itemtype="gpcm" such as in the case of mirt function? Thank you very much!

@detful83 2 жыл бұрын

Amazing and useful video, thank you! One question: why fstruct3 only included items from G and F1 but not F2, when the previously tested bifactor model included items from both F1 and F2?

@statsguidetree 2 жыл бұрын

It was just a decision to use items from the factor with the higher value loadings in fstruct2 for a new example in fstruct3. It was just for an example and necessarily a method you should follow when exploring data.

@leondbleondb 3 жыл бұрын

Thank you very much for this video. Much appreciated. Can one also test for nonuniform DIF with mirt? I assume it also has a plot option to visualize any DIF?

@statsguidetree 3 жыл бұрын

Yes, In the DIF() function include the intercept and slope parameters for nonuniform. You can do a general visualization by including your multipleGroup() in the plot() function e.g. plot(multipleGroup()). Though it is harder to visualize with multidimensionality. There is an option to produce a general plot for specific items. For example to look for item1, itemplot(multipleGroup(), 1).

@littleheavenonearth86 2 ай бұрын

very informative. Can I use this MIRT for an instrument with more than 2 dimensions? I am working on validating a research instrument that is polytomous with 4 dimensions but 1 latent trait.

@smritimehta 3 жыл бұрын

This is extremely helpful, thank you! Question: If an instrument has two theoretically driven dimensions but a subset of the items are expected to load onto both dimensions, is there a way to account for that using mirt?

@statsguidetree 2 жыл бұрын

Thank you for the compliment. To your question: yes, the package allows for different factors to be mapped to the same items. You can do so when specifying your confirmatory model or if you are unsure you can run the exploratory MIRT and observe which items are loaded on to the factors you specified. I hope that helped.

@nicolem9516 2 жыл бұрын

Dear professor, I loved your video. I just have a question. In the minute <a href="#" class="seekto" data-time="2125">35:25</a> you mention that you have to be "certain that these items are free of" and I don't understand the next word you mentioned. Is it "diff" as the close caption suggests of "depth", or something else?

@statsguidetree 2 жыл бұрын

The statement should be "free of diff", which means that the items do not function differentially. The items you set for your anchors should not differ across groups. Usually you would need some content knowledge of the topic the items are based on or some prior research to determine this.

@markusreiser184 2 жыл бұрын

I am dealing with a 3 Factor (multidimensional) Test (dichotomous data) I'd like to validate. Since I have just started experiencing R my questions are rather of very basic nature : 1. How do I get the item output- especially the infit and outfit output and item difficulty and discrimination . The item info function showed different output. 2. How can a Wrightmap or a factor model be plotted.... Has anyone got an idea?

@metalslegend Жыл бұрын

when you type IRTpars = TRUE, simplify = TRUE in the coef() command, you will receive the actual threshold parameters b. The intercepts cannot be interpreted properly.

@WARStPierre 2 жыл бұрын

Thanks so much for this. This is a topic I requested so I really appreciate it. One question though, I know effect size takes on more importance with larger sample sizes, with varying guidelines on how many logits difference indicates DIF. How do we check the effect size of the DIF so that we are not only relying on statistical significance?

@statsguidetree 2 жыл бұрын

My apologies for the delayed response. So, to my understanding of the package effect sizes for DIF isn't offered. However with a p.adjust you can add a multiplicity control to mitigate the issue of many likelihood ratio tests being conducted.

@tamyrestomazpaiva1618 2 жыл бұрын

Good explanation. But could you make the script available to test in my analyses? Thank you very much.

@statsguidetree 2 жыл бұрын

I put the rcode into a pinned comment. I will include it here to for you to copy and paste: ################################################################################ ########## overview # Multidimensional Item Response Theory (MIRT) models # Exploratory/confirmatory MIRT models # Bifactor models # DIF with anchors # Multidimesnional IRT models (i.e., more than one factor) are generalizations # Traditional unidimensional IRT models (e.g., 1PL, 2PL, 3PL, GRM, etc.) # can each be gneralized to estimate additional dimensions (i.e., factors) # These Multidiemsional IRT models can be confirmatory (i.e., number of factors # in the scale are known prior) or exploratory (i.e., number of factors in # the scale are not known prior). ################################################################################ ########## dataset # Generic Conspiracist Beliefs Scale (GCBS; Brotherton, French, # and Pickering, 2013) response dataset. # Source: openpsychometrics.org/_rawdata/ # Load dataset downloaded 15 items with scale containing 5 levels # (1=S.Disagree,2=Disagree, 3=Neutral, 4=Agree, and 5=S.Agree) gcbs

@bayunoest 2 жыл бұрын

Thank you for the explanation in this topic. Could you enlighten me about second-order model using mirt package? Because i'm curious about that. Is there any r code for second-order model using mirt?

@statsguidetree 2 жыл бұрын

For the second order model, do you happen to be referring to the bi-factor. If not could you provide some more detail ?

@bayunoest 2 жыл бұрын

@@statsguidetree yes, bifactor model and testlet model. Is the bifactor model the same as the second order model like in CFA?

@statsguidetree 2 жыл бұрын

@@bayunoest I had to do some investigating to make sure I am not interpreting your question incorrectly. It could be that different sources use different names. But, a bi-factor model is when you have all items loaded on to one factor and some items loaded also on a subsequent factor. This is different from a two factor model where items in a test are either mostly loaded on factor 1 or factor 2. I hope that was helpful.

@andimuhyi8307 2 жыл бұрын

Excuse me, sir How to increase limited row ....at command options(max.print= 99999) I just get 160 row

@statsguidetree 2 жыл бұрын

I am not too familiar with this. You can try running the line options(max.print=10000) before running your function. But, if you trying to view a dataset, you may want to save it as a data frame with as.data.frame() then open it up with view().

@andimuhyi8307 2 жыл бұрын

Ok I'll try, thanx so much ... 🙏

@cristianechammas5355 2 жыл бұрын

Please, can I fit a Thurstonian IRT Model in R? With mirt?

@statsguidetree 2 жыл бұрын

To my knowledge it is not available in the mirt package. But, there is a separate package in R available for it called thursonianIRT. Here is a link to the package cran.r-project.org/web/packages/thurstonianIRT/thurstonianIRT.pdf

@cristianechammas5355 2 жыл бұрын

@@statsguidetree Thank you very much :-)

@elloisejackson2398 Жыл бұрын

Why do we suppress it to 0.25 when looking at the factors?

@andrej.mentel Жыл бұрын

It's more or less just a convention for the sake of clarity of results. Personally, I prefer the 0.3 cut-off. This can be interpreted to mean that a factor with a loading of < 0.25 on a particular item can be considered a "factor of negligible influence". However, remember that this is an exploratory analysis, so we use this to estimate what the possible structure might be. In the confirmatory case, we explicitly assume that (if the particular item is an indicator of just one factor) the factor loading of the other factor is zero.

@sarahwtz328 2 жыл бұрын

What is an anchor item

@statsguidetree 2 жыл бұрын

Anchor items are if you are using the anchoring method. These are items we identify as being DIF free (i.e., non-bias items) that we can use as a point of comparison to detecting DIF among other items.