added fit by group table #145

LSLindeloo · 2023-02-17T11:52:33Z

jasp-stats/jasp-issues#711 (fixed for SEM)
fixes jasp-stats/jasp-issues#2690
also adds chi^2 diff tests for multiple models and groups

Kucharssim

looks great, just a couple of comments

Kucharssim · 2023-03-01T10:56:47Z

R/sem.R

@@ -203,7 +203,7 @@ checkLavaanModel <- function(model, availableVars) {
    modelContainer <- createJaspContainer()
    modelContainer$dependOn(c("samplingWeights", "meanStructure", "manifestInterceptFixedToZero", "latentInterceptFixedToZero", "exogenousCovariateFixed", "orthogonal",
                              "factorScaling", "residualSingleIndicatorOmitted", "residualVariance", "exogenousLatentCorrelation",
-                              "dependentCorrelation", "threshold", "scalingParameter", "efaConstrained", "standardizedVariable", "naAction", "estimator", "test",
+                              "dependentCorrelation", "threshold", "scalingParameter", "efaConstrained", "standardizedVariable", "naAction", "estimator", "modelTest",


Why do we change it here? Was this a bug?

Kucharssim · 2023-03-01T10:56:57Z

R/sem.R

@@ -360,11 +360,11 @@ checkLavaanModel <- function(model, availableVars) {
  lavopts[["estimator"]]   <- options[["estimator"]]
  lavopts[["se"]]          <- ifelse(options[["errorCalculationMethod"]] == "bootstrap", "standard", options[["errorCalculationMethod"]])
  lavopts[["information"]] <- options[["informationMatrix"]]
-  lavopts[["test"]]        <- ifelse(options[["modelTest"]] == "satorraBentler", "Satorra.Bentler",
-                                     ifelse(options[["modelTest"]] == "yuanBentler", "Yuan.Bentler",
+  lavopts[["test"]]        <- ifelse(options[["modelTest"]] == "satorraBentler", "satorra.bentler",


The same here, was this a bug?

Kucharssim · 2023-03-01T10:57:31Z

R/sem.R

+    grouptab$addColumnInfo(name = "AIC",      title = gettext("AIC"),                type = "number" )
+    grouptab$addColumnInfo(name = "BIC",      title = gettext("BIC"),                type = "number" )
+    grouptab$addColumnInfo(name = "N",        title = gettext("n"),                  type = "integer")
+    grouptab$addColumnInfo(name = "Chisq",    title = gettext("&#967;&sup2;"),       type = "number" ,


Suggested change

grouptab$addColumnInfo(name = "Chisq", title = gettext("χ²"), type = "number" ,

grouptab$addColumnInfo(name = "Chisq", title = "\u03C7\u00B2", type = "number" ,

Kucharssim · 2023-03-01T10:57:57Z

R/sem.R

+    grouptab$addColumnInfo(name = "PrChisq",  title = gettext("p"),                  type = "pvalue",
+                         overtitle = gettext("Baseline test"))
+    if (length(options[["models"]]) > 1) {
+      grouptab$addColumnInfo(name = "dchisq",   title = gettext("\u0394\u03C7\u00B2"), type = "number" ,


Suggested change

grouptab$addColumnInfo(name = "dchisq", title = gettext("\u0394\u03C7\u00B2"), type = "number" ,

grouptab$addColumnInfo(name = "dchisq", title = "\u0394\u03C7\u00B2", type = "number" ,

if it's only symbols we don't need to translate

Kucharssim · 2023-03-01T11:00:03Z

R/sem.R

+    if (length(options[["models"]]) > 1) {
+      grouptab$addColumnInfo(name = "dchisq",   title = gettext("\u0394\u03C7\u00B2"), type = "number" ,
+                           overtitle = gettext("Difference test"))
+      grouptab$addColumnInfo(name = "ddf",      title = gettext("\u0394df"),           type = "integer",


here we only want to translate the "df"

Suggested change

grouptab$addColumnInfo(name = "ddf", title = gettext("\u0394df"), type = "integer",

grouptab$addColumnInfo(name = "ddf", title = gettextf("%1$sdf", "\u0394"), type = "integer",

juliuspfadt · 2024-02-21T15:32:47Z

juliuspfadt · 2024-02-21T16:28:09Z

although this seems wrong to me, why would the same model with fewer Ns have fewer Df?

juliuspfadt · 2024-02-21T16:32:45Z

It also seems very annoying that we have to fit the model again for each group, if only lavaan would give us the AIC and BIC

juliuspfadt · 2024-02-21T16:58:41Z

R/sem.R

+        equalityConstraints <- syntax[syntax$op == "==",]
+        if (nrow(equalityConstraints) > 0) {
+          for (j in 1:nrow(equalityConstraints)) {
+            if(equalityConstraints[j, "lhs"] %in% syntax_group[, "plabel"]) {
+              syntax_group <- rbind(syntax_group, equalityConstraints[j,])
+            }
+          }
+        }
+        syntax_group[["group"]] <- syntax_group[["block"]] <- rep(1, nrow(syntax_group))
+        lav_args[["model"]] <- syntax_group
+        dataset_group <- dataset[dataset[[options[["group"]]]] == groupNames[group],]
+        lav_args[["data"]]  <- dataset_group
+        fit_group <- try(do.call(lavaan::lavaan, lav_args))


@Kucharssim, this code does not work when there are equality constraints, which is not surprising. Lorenzo fits the model for the subgroup only, but with invariance testing equality constraints are between the two groups, e.g., the loadings of group1 are equal to the loadings of group2. It seems to me that the equality constraints do not matter for the fit of each group, so we should estimate the model with free parameters unless some the constraints are not between groups but within a group. I wonder how we would find distinguish here though....

juliuspfadt · 2024-02-23T13:39:58Z

Okay, this should be ready as well. this is what I eventually settled on:

juliuspfadt · 2024-04-30T17:37:06Z

now like this:

Kucharssim

The functionality looks really good, I didn't find anything suspicious in the code...

The tests fail spectacularly though, can you look into that? It seems to me these failures could be introduced by the PR, so it would be good to polish that before merging

…e groups

juliuspfadt · 2024-05-17T10:15:22Z

I suppose I should've checked, weird that I didn't notice the test failing. It's caused by imposing meanstructure but then not fixing any intercepts to 0. I wonder if in qml there is an option for something like a triple of checkboxes, they can be all checked but not unchecked simultaneously. Suppose .quitAnalysis will have to do for now.

LSLindeloo requested a review from Kucharssim February 17, 2023 11:52

Kucharssim requested changes Mar 1, 2023

View reviewed changes

juliuspfadt force-pushed the chisqPerGroup branch from 6daf3fe to 9ad5223 Compare February 21, 2024 13:08

juliuspfadt requested a review from Kucharssim February 21, 2024 13:11

juliuspfadt force-pushed the chisqPerGroup branch from 9ad5223 to a1a2d04 Compare February 21, 2024 13:45

juliuspfadt force-pushed the chisqPerGroup branch from a1a2d04 to f10573d Compare February 21, 2024 15:52

juliuspfadt reviewed Feb 21, 2024

View reviewed changes

juliuspfadt force-pushed the chisqPerGroup branch from 6b9c520 to a6c1b7d Compare February 23, 2024 13:26

juliuspfadt force-pushed the chisqPerGroup branch from 0531f85 to 9a90250 Compare April 30, 2024 17:31

juliuspfadt force-pushed the chisqPerGroup branch 2 times, most recently from ce79b5d to 5332b4e Compare April 30, 2024 17:39

Kucharssim reviewed May 16, 2024

View reviewed changes

juliuspfadt added 7 commits May 17, 2024 11:50

merge conflicts

da58cf1

small changes requested by simon

6498108

change layout of multigroup fit table

f6b85c9

try fixing the tests, but more work to do

cc83c77

fit per group needs more work

2016c9d

model fit by group table finish

4483550

fix multi model fit table bug, and add chisq diff teststs for multipl…

f7c732e

…e groups

juliuspfadt force-pushed the chisqPerGroup branch from 5332b4e to f7c732e Compare May 17, 2024 09:50

fix the test

7912e99

catch error with meanstructure and no intercepts fixed to zero

fed8b7e

juliuspfadt force-pushed the chisqPerGroup branch from b09b7c3 to 6fda55c Compare May 18, 2024 14:24

try fixing failing tests

588e4e5

juliuspfadt force-pushed the chisqPerGroup branch from 6fda55c to 588e4e5 Compare May 21, 2024 08:14

Kucharssim merged commit e065778 into jasp-stats:master May 21, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added fit by group table #145

added fit by group table #145

LSLindeloo commented Feb 17, 2023 •

edited by juliuspfadt

Loading

Kucharssim left a comment

Kucharssim Mar 1, 2023

Kucharssim Mar 1, 2023

Kucharssim Mar 1, 2023

Kucharssim Mar 1, 2023

Kucharssim Mar 1, 2023

juliuspfadt commented Feb 21, 2024 •

edited

Loading

juliuspfadt commented Feb 21, 2024

juliuspfadt commented Feb 21, 2024

juliuspfadt Feb 21, 2024

juliuspfadt commented Feb 23, 2024 •

edited

Loading

juliuspfadt commented Apr 30, 2024

Kucharssim left a comment

juliuspfadt commented May 17, 2024

	grouptab$addColumnInfo(name = "Chisq", title = gettext("χ²"), type = "number" ,
	grouptab$addColumnInfo(name = "Chisq", title = "\u03C7\u00B2", type = "number" ,

	grouptab$addColumnInfo(name = "dchisq", title = gettext("\u0394\u03C7\u00B2"), type = "number" ,
	grouptab$addColumnInfo(name = "dchisq", title = "\u0394\u03C7\u00B2", type = "number" ,

	grouptab$addColumnInfo(name = "ddf", title = gettext("\u0394df"), type = "integer",
	grouptab$addColumnInfo(name = "ddf", title = gettextf("%1$sdf", "\u0394"), type = "integer",

added fit by group table #145

added fit by group table #145

Conversation

LSLindeloo commented Feb 17, 2023 • edited by juliuspfadt Loading

Kucharssim left a comment

Choose a reason for hiding this comment

Kucharssim Mar 1, 2023

Choose a reason for hiding this comment

Kucharssim Mar 1, 2023

Choose a reason for hiding this comment

Kucharssim Mar 1, 2023

Choose a reason for hiding this comment

Kucharssim Mar 1, 2023

Choose a reason for hiding this comment

Kucharssim Mar 1, 2023

Choose a reason for hiding this comment

juliuspfadt commented Feb 21, 2024 • edited Loading

juliuspfadt commented Feb 21, 2024

juliuspfadt commented Feb 21, 2024

juliuspfadt Feb 21, 2024

Choose a reason for hiding this comment

juliuspfadt commented Feb 23, 2024 • edited Loading

juliuspfadt commented Apr 30, 2024

Kucharssim left a comment

Choose a reason for hiding this comment

juliuspfadt commented May 17, 2024

LSLindeloo commented Feb 17, 2023 •

edited by juliuspfadt

Loading

juliuspfadt commented Feb 21, 2024 •

edited

Loading

juliuspfadt commented Feb 23, 2024 •

edited

Loading