Regarding the results at the end involving children, it's important to note that it can't distinguish relative effects from absolute effects. That is, one can't predict that if you increased schooling for everyone it would have the same effect. I don't think you were claiming this, but I just think that when you try and relate those results back to a useful prediction it's complicated.
That's right and a good point. These predictions are always *retrospective*, meaning they are in the existing environment (or the environment of the previous generation since biobank participants are typically adults). If you change the phenotype for many people through some intervention you are by definition changing the environment.
"Whereas genetics tells us there’s nothing special about g and we should probably ditch the common cause / general factor model altogether."
Crazily wrong take. Genomic SEM tells us that g is *not the only broad factor that exists*; however it also tells us that it does exist, in the sense that you've got multiple genetic variants "on g".
You have a bad habit of comparing the strength to which one thing exists against the strength to which another thing exists, and rejecting it if it doesn't vastly exceed it. E.g. you tried the same with race.
As an analogy: at work I can perform factor analysis on our performance telemetry. One of the factors I get turns out to correspond to CPU exhaustion. This isn't the only factor; there also seems to be stuff related to the database and the network. But it would be crazy for me to assert that because the CPU doesn't exhibit a much strongly statistical signal than the database, we should ditch the CPU from our models of how program performance goes.
Genetic associations alone cannot tell us whether something does or doesn't exist. What the genetics is telling us is: (1) g and IQ have nearly perfect genetic correlation and equivalent heritabilities and are thus essentially indistinguishable genetic constructs ("there's nothing special about g"); and (2) network models that do not assume a common cause both fit the data consistently better and produce vastly different genetic estimates ("we should probably ditch the common cause model").
There is probably a near-perfect genetic correlation between g and IQ, but you should surely know that working with an "extracted g factor" is not the way to test it. "Extracted g factor" is just another weighting for an IQ test, and if you wanted to estimate the correlation from statistics like that, you should instead work with something like omega hierarchical.
But it's a moot point because the more relevant opportunity to genomics is that you can "purify" the g-factor by finding *variants* on g vs other variants. You yourself linked a study showing that some variants had effects that were not consistent with g and other variants had effects that were consistent with g, so clearly there's some g vs non-g distinction even at the genomic level. And yes, these inconsistent variants can be cross-cutting too, which is in contradiction to usual g models, but that is an opportunity rather than a problem: by learning the patterns in which they are cross-cutting, we can "break up" IQ tests into some different measures that hopefully better cut the world at its joints.
Could you be a bit more clear by what you mean by suggesting we ditch the common cause/general factor model all together?
I mean we certainly know there are common causes in the sense that someone who experiences serious nutritional of other deprivation during certain stages does perform worse on the full range of mental ability tests. So I doubt you are suggesting we need to ditch the idea that there can be common causes in this area. But then I don't quite understand what the claim amounts to?
Is it merely a claim about a better fit by one kind of model then another? If so, is there any reason to derive any kind of inference from this about the way the world works or is it purely an issue of modeling practice?
I'm specifically talking about the interpretation of a general factor as a unitary common cause. We see genetic variants that are associated with g but not with many skills and variants that are not associated with g but with many skills; so "effect on g" is neither necessary nor sufficient to influence many skills. Then we see that network models provide a better fit to the data (both in sample and out of sample) than a general factor model, so g is also not an effective way to explain patterns in data relative to other models.
So do you mean this cashes out to the claim that models which represent the situation in terms of multiple skills and connections between them provide a better fit than a model where any correlation between the skills is mediated via a single common factor g? But isn't that always guaranteed? I mean, any effect of a common cause can always be pushed out into direct relations between the observed variables.
Isn't this like arguing there is no effect of teacher quality on student test scores because we can get a better fit to data using models which replace a teacher quality variable with a direct effect of student performance on the performance of other students in the same class? After all, the teacher quality model constrains the effect to be the same for each class while postulating an effect of students on other students in the same class is effectively just postulating some other unknown common effect per class.
This is not always guaranteed, and certainly not when evaluating model fit in a completely held-out sample. I recommend taking a look at Kan et al. 2019 (https://linkinghub.elsevier.com/retrieve/pii/S0160289618300114) which considers this question and also performs simulations showing the network model does not fit better when the true generative model is a common factor. I also summarize this and a few other studies here: http://gusevlab.org/projects/hsq/#h.uirpqwnbalnc if you are interested in more of the weeds.
Sorry, you are correct and I wasn't sufficently clear. Yes, obviously, we have good reason to disfavor the explanation that the *only* way that correlation between the factors results is via the mediation of a simple single common cause g.
But no one actually believes this is really the case. That would be crazily over simple to deny that there could be any correlation not through g at all. Everyone accepts that the true story in the brain is going to be more complicated than that of either model so it's not that relevant to point to the fact that the simple single cause model better fits held out data generated by that process.
I mean, if I generate data by choosing x via some distribution and then generating y as mx+c+ N(0,1) (or last term choosen from that distribution) then I should expect that linear regression will better fit held out data than a quadratic model even though the later strictly extends the former. In effect, you've given the model the hint that the quadratic term is 0 which obviously helps fit. But, even if the relationship is primarily linear but has some tiny non-linear effect then the quadratic fit will start to work better on the held out data (and just having different error distributions may be enough). After all, we know that our models aren't really perfect but treat them as good enough.
Here the question isn't can there be any non-zero correlation not mediated by g but whether the g model is a sufficiently strong explanation that we should be looking for some kind of shared cause out there in the world and/or that it's a useful simplifying assumption when working with data sets too small to pin down all the parameters in the larger models.
As your first link correctly points out, identifying the right explanation involves balancing fit with Poperian/Occam simplicity considerations.
--
Look, my intuition is that g is overrated especially for the kinds of things that most people care about but I don't think you're argument is sufficient to reject g as part of the best explanation at some given level of detail. Whether or not you want to model it via network or other model is a different question than the best explanation.
Hell, at a really trivial level there are some rare common cause effects (mutations that affect all nuerons) so the truly best explanation must include some common causes.
I would suggest reading the Kan et al. paper (here's a sci-hub link in case you have issues accessing it: https://sci-hub.se/https://www.sciencedirect.com/science/article/abs/pii/S0160289618300114?via%3Dihub) or the Knyspel + Plomin paper (https://www.sciencedirect.com/science/article/pii/S0160289624000278) I cited in the post. As you note, interpreting model fit is not trivial because you need to be aware of in-sample vs out-of-sample fit, parsimony, and theoretical plausibility -- which these papers discuss in detail. In particular, Kan et al. fit a variety of different factor-based models (with g as a common cause in some part of the model), including some that have more degrees of freedom than the network model, and show superior performance with network models. In some cases the performance is substantially better such that factor models would be considered a poor fit and network models a good fit based on conventions in the field. Knyspel + Plomin also show the network models produces negative relationships between subtests which are untenable under the factor model and that these negative relationships are supported by the genetic data (they argue that both types of models can potentially be useful though). Both papers are written by esteemed psychometricians/psychologists and discuss this issue thoughtfully and in detail.
Also, in my experience quite a few people believe that a simple and causal general factor model is a basically settled science and get either confused or angry when shown that other model structures can fit the data as well or better.
I should have specified in my example above that I intended for there to be a non-zero true effect of students on other students in the class. And for us to know there is some fixed number of teachers but not who teaches each class.
Thus, for sufficiently large data sets, we should expect the student effect model to outperform the teacher model on held out data (for sufficiently large classes we will converge to the true class effect subsuming the teacher effect) assuming the held out data is always a small subset of each class. (yes if you use the model to predict some previously unobserved class it's different...but that's analagous to using a previously unseen kind of mental test which isn't common).
Exactly what makes the per teacher effect part of the best explanation is that this model has less degrees of freedom (only allows number of teacher different buckets for classes not an arbitrary per class effect) but is almost as good as the model with more -- though the complete best explanation will involve a per class effect as well.
Regarding the results at the end involving children, it's important to note that it can't distinguish relative effects from absolute effects. That is, one can't predict that if you increased schooling for everyone it would have the same effect. I don't think you were claiming this, but I just think that when you try and relate those results back to a useful prediction it's complicated.
That's right and a good point. These predictions are always *retrospective*, meaning they are in the existing environment (or the environment of the previous generation since biobank participants are typically adults). If you change the phenotype for many people through some intervention you are by definition changing the environment.
"Whereas genetics tells us there’s nothing special about g and we should probably ditch the common cause / general factor model altogether."
Crazily wrong take. Genomic SEM tells us that g is *not the only broad factor that exists*; however it also tells us that it does exist, in the sense that you've got multiple genetic variants "on g".
You have a bad habit of comparing the strength to which one thing exists against the strength to which another thing exists, and rejecting it if it doesn't vastly exceed it. E.g. you tried the same with race.
As an analogy: at work I can perform factor analysis on our performance telemetry. One of the factors I get turns out to correspond to CPU exhaustion. This isn't the only factor; there also seems to be stuff related to the database and the network. But it would be crazy for me to assert that because the CPU doesn't exhibit a much strongly statistical signal than the database, we should ditch the CPU from our models of how program performance goes.
Genetic associations alone cannot tell us whether something does or doesn't exist. What the genetics is telling us is: (1) g and IQ have nearly perfect genetic correlation and equivalent heritabilities and are thus essentially indistinguishable genetic constructs ("there's nothing special about g"); and (2) network models that do not assume a common cause both fit the data consistently better and produce vastly different genetic estimates ("we should probably ditch the common cause model").
There is probably a near-perfect genetic correlation between g and IQ, but you should surely know that working with an "extracted g factor" is not the way to test it. "Extracted g factor" is just another weighting for an IQ test, and if you wanted to estimate the correlation from statistics like that, you should instead work with something like omega hierarchical.
But it's a moot point because the more relevant opportunity to genomics is that you can "purify" the g-factor by finding *variants* on g vs other variants. You yourself linked a study showing that some variants had effects that were not consistent with g and other variants had effects that were consistent with g, so clearly there's some g vs non-g distinction even at the genomic level. And yes, these inconsistent variants can be cross-cutting too, which is in contradiction to usual g models, but that is an opportunity rather than a problem: by learning the patterns in which they are cross-cutting, we can "break up" IQ tests into some different measures that hopefully better cut the world at its joints.
It looks like you just blew right past what I said and went on to argue two new things.
Not two new things; I expanded on my original thing because you hadn't understood what I was originally getting at.
Seconded. This sentence really jumped out of the left corner after the previous text.
Could you be a bit more clear by what you mean by suggesting we ditch the common cause/general factor model all together?
I mean we certainly know there are common causes in the sense that someone who experiences serious nutritional of other deprivation during certain stages does perform worse on the full range of mental ability tests. So I doubt you are suggesting we need to ditch the idea that there can be common causes in this area. But then I don't quite understand what the claim amounts to?
Is it merely a claim about a better fit by one kind of model then another? If so, is there any reason to derive any kind of inference from this about the way the world works or is it purely an issue of modeling practice?
I'm specifically talking about the interpretation of a general factor as a unitary common cause. We see genetic variants that are associated with g but not with many skills and variants that are not associated with g but with many skills; so "effect on g" is neither necessary nor sufficient to influence many skills. Then we see that network models provide a better fit to the data (both in sample and out of sample) than a general factor model, so g is also not an effective way to explain patterns in data relative to other models.
So do you mean this cashes out to the claim that models which represent the situation in terms of multiple skills and connections between them provide a better fit than a model where any correlation between the skills is mediated via a single common factor g? But isn't that always guaranteed? I mean, any effect of a common cause can always be pushed out into direct relations between the observed variables.
Isn't this like arguing there is no effect of teacher quality on student test scores because we can get a better fit to data using models which replace a teacher quality variable with a direct effect of student performance on the performance of other students in the same class? After all, the teacher quality model constrains the effect to be the same for each class while postulating an effect of students on other students in the same class is effectively just postulating some other unknown common effect per class.
This is not always guaranteed, and certainly not when evaluating model fit in a completely held-out sample. I recommend taking a look at Kan et al. 2019 (https://linkinghub.elsevier.com/retrieve/pii/S0160289618300114) which considers this question and also performs simulations showing the network model does not fit better when the true generative model is a common factor. I also summarize this and a few other studies here: http://gusevlab.org/projects/hsq/#h.uirpqwnbalnc if you are interested in more of the weeds.
Sorry, you are correct and I wasn't sufficently clear. Yes, obviously, we have good reason to disfavor the explanation that the *only* way that correlation between the factors results is via the mediation of a simple single common cause g.
But no one actually believes this is really the case. That would be crazily over simple to deny that there could be any correlation not through g at all. Everyone accepts that the true story in the brain is going to be more complicated than that of either model so it's not that relevant to point to the fact that the simple single cause model better fits held out data generated by that process.
I mean, if I generate data by choosing x via some distribution and then generating y as mx+c+ N(0,1) (or last term choosen from that distribution) then I should expect that linear regression will better fit held out data than a quadratic model even though the later strictly extends the former. In effect, you've given the model the hint that the quadratic term is 0 which obviously helps fit. But, even if the relationship is primarily linear but has some tiny non-linear effect then the quadratic fit will start to work better on the held out data (and just having different error distributions may be enough). After all, we know that our models aren't really perfect but treat them as good enough.
Here the question isn't can there be any non-zero correlation not mediated by g but whether the g model is a sufficiently strong explanation that we should be looking for some kind of shared cause out there in the world and/or that it's a useful simplifying assumption when working with data sets too small to pin down all the parameters in the larger models.
As your first link correctly points out, identifying the right explanation involves balancing fit with Poperian/Occam simplicity considerations.
--
Look, my intuition is that g is overrated especially for the kinds of things that most people care about but I don't think you're argument is sufficient to reject g as part of the best explanation at some given level of detail. Whether or not you want to model it via network or other model is a different question than the best explanation.
Hell, at a really trivial level there are some rare common cause effects (mutations that affect all nuerons) so the truly best explanation must include some common causes.
I would suggest reading the Kan et al. paper (here's a sci-hub link in case you have issues accessing it: https://sci-hub.se/https://www.sciencedirect.com/science/article/abs/pii/S0160289618300114?via%3Dihub) or the Knyspel + Plomin paper (https://www.sciencedirect.com/science/article/pii/S0160289624000278) I cited in the post. As you note, interpreting model fit is not trivial because you need to be aware of in-sample vs out-of-sample fit, parsimony, and theoretical plausibility -- which these papers discuss in detail. In particular, Kan et al. fit a variety of different factor-based models (with g as a common cause in some part of the model), including some that have more degrees of freedom than the network model, and show superior performance with network models. In some cases the performance is substantially better such that factor models would be considered a poor fit and network models a good fit based on conventions in the field. Knyspel + Plomin also show the network models produces negative relationships between subtests which are untenable under the factor model and that these negative relationships are supported by the genetic data (they argue that both types of models can potentially be useful though). Both papers are written by esteemed psychometricians/psychologists and discuss this issue thoughtfully and in detail.
Also, in my experience quite a few people believe that a simple and causal general factor model is a basically settled science and get either confused or angry when shown that other model structures can fit the data as well or better.
I should have specified in my example above that I intended for there to be a non-zero true effect of students on other students in the class. And for us to know there is some fixed number of teachers but not who teaches each class.
Thus, for sufficiently large data sets, we should expect the student effect model to outperform the teacher model on held out data (for sufficiently large classes we will converge to the true class effect subsuming the teacher effect) assuming the held out data is always a small subset of each class. (yes if you use the model to predict some previously unobserved class it's different...but that's analagous to using a previously unseen kind of mental test which isn't common).
Exactly what makes the per teacher effect part of the best explanation is that this model has less degrees of freedom (only allows number of teacher different buckets for classes not an arbitrary per class effect) but is almost as good as the model with more -- though the complete best explanation will involve a per class effect as well.