Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

How about pool error term for RM factors?

Hi,

When running a RMANOVA it seems like pool error term for RM factors is pre-checked. Could somebody enlighten me in which circumstances to check it and when NOT to have it checked? 🙄

Thank you for guidance:-)

Per


Comments

  • The help file refers to Morey (2008) for details. I also Googled this and also found people who believe it is prudent not to pool the error term (e.g., http://people.umass.edu/~bogartz/on_pooling_error_terms..html). I'll ask Johnny for his input.

    Cheers,

    E.J.

  • Hi @PerPalmgren,

    The initial RM ANOVA model is a model where the error terms are pooled - this is simply how linear models are set up in these settings. As a consequence, the emmeans R-package (which we use for marginal means, contrasts, and posthoc tests) also works with the pooled error term (see the documentation here).

    You can read further about this in a previous thread:

    That's how the emmeans works: it operates based on the model you specify. In the ANOVA model, there is the assumption of equal variances, so that influences how emmeans handles the pairwise comparisons. We offer a workaround, where (for balanced designs only) in the RM ANOVA you can choose to use the non-pooled error term for RM factors. This basically comes down to running a series of paired samples t-tests.

    I am currently trying to have the anova be able to specify different types of models (i.e., gls), so that we can offer this possibility while still using the lovely emmeans package.

    see for instance https://stats.stackexchange.com/questions/369532/interpreting-the-standard-error-from-emmeans-r

    I feel like the default behavior of emmeans ought to be respected (it makes sense to have pooled errors for followup tests if the original model is also using the pooled error term), hence the pooled error term is the default option. Does this clarify the issue for you?

    Kind regards,

    Johnny

  • Hi @JohnnyB ,


    Would you also advise pool error terms for simple main effects? E.g., in a RM ANOVA, you have a sig. interaction between the RM factor (A) and the between-subjects factor (B) and you follow-up this interaction with simple main effects in which A is the simple effect factor and B the moderator.

    Thanks!

    Tomás

  • Hi Tomas,

    Yes, I would use the pooled error term by default, since that is in line with the anova model. Only if you have substantial grounds for doing otherwise, I would switch to unpooled.

    Kind regards

    Johnny

  • edited June 2022

    @JohnnyB When reading the documentation you provided and the article by Bogartz https://people.umass.edu/~bogartz/on_pooling_error_terms..html provided by @EJ I am not 100% convinced of the option of employing pooled error terms. I think it would be easier for JASP users to go with one or the other. One could than in a scientific discourse or statistical debate argue for that this is how the post hoc calculation is done in the statistical program. I think the present choice is very confusing. Further, the article Morey (2008) which is referenced in JASP does not adress the issue of pooled errors.

  • 1. Why is non-pooling only an option for the repeated-factors and not for the between-subject factors?

    2. The reason I think non-pooling is necessary is that there should be some way to conduct planned comparisons without having those comparisons be part of an ANOVA. Perhaps jamovi could be modified to include a planned-comparisins analysis with no error-term pooling, rather than include non-pooling as an option in the ANOVA analysis.

    R

  • @PerPalmgren, I am not sure if both discuss the same issue. It seems the post you reference speaks about the main model in ANOVA, whereas above we were discussing the best course for a follow-up test on a default ANOVA, but I could be mistaken. I feel that the follow-up tests should follow the main model as closely as possible (see this lengthy discussion here, with the author of the emmeans package outlining their arguments). If the default ANOVA is wrong to begin with (as seems to be implied by Bogartz?), a different starting model should be used, such as a GLM.

    However, if it turns out that the Bogartz finding also holds for the pooled SE of post hoc tests on a default ANOVA, then maybe we should indeed reconsider the default value for that option.

    @andersony3k

    1. For the reason outlined above, it has not been high priority to also implement it for between subjects (In most cases where the non-pooled SE seems to the more sensible option, this concerns within subjects factors).
    2. We recently implemented generalized linear models in the Regression module, where you can specify contrasts as well. This might be what you are looking for (note that this module requires the data to be in long data format, whereas the RM ANOVA requires wide data format).


    Kind regards,

    Johnny

  • For #2. Sometimes the data severely violate the assumptions of a linear model, and so the analyst would like to do a set of planned t tests (instead of fitting the data to a linear model such as an ANOVA or a Generalized Linear Model). JASP supports this if one is willing to do the t tests one at a time. It would be convenient of one could present JASP with a set of factors and have it automatically perform a set of individual (not linear-model-based) t tests.

    R

  • Hi @andersony3k ,

    Yes that would be a great functionality! We are now exploring the possibilty of specifying a more flexible structure for the SE's (with the nlme package) in ANOVA, which would make this functionality very straightforward to implement. In the meantime, I'm afraid you are stuck with specifying the individual t-tests in the t-test module (a bit of clicking, but wont take that long I reckon).

    Kind regards,

    Johnny

  • edited April 2023

    @JohnnyB

    Hi,

    Despite being JASP lover and of the opinion that JASP is the most educative and stringent statisitcal program on the market, I still do not agree with using the "Pool error term for RM factors" as the default for posthoc analysis with a RM-ANOVA. Recently in a workshop for doctoral students whom where using both JASP. SPSS and Jamovi there was a lot of confusion as the results differed between the programs (all using the default settings). In my opinion one imperative aspect of science is to to strive for better collective knowledge and understanding of different phenomena of the world. One way is to create a common language and practice. I think JASP would better contribute to that discourse if the setting was harmonized. I understand your opinion (based on above) of using the pool term is to prefer. However, to create coherency in the scientific community, I think JASP would do better to have the Pool error unchecked as default or make information should be much more clear to the user. As pragmatic user I think ii is more important to have comparable and analogous results between programs, especially with regards to P-values and measures of effect sizes.

    Please see attached outputs from JASP, SPSS and Jamovi.

    All the best

    Per


  • Did also the post hoc in DATAtab this morning (using default) and got:


  • I think we could probably add a note underneath the table. On the other hand, I am not married to this particular setting, and if most programs have a different default then I don't see why we ought to stick religiously to the current setting.

    E.J.

  • edited April 2023

    @PerPalmgren

    Note however that there's currently more consistency *within* JASP than there is within SPSS and within jamovi regarding error-term pooling in repeated-measures versus between-subject ANOVA post hoc tests. In SPSS an jamovi, the post-hocs for a between-subject use pooled error terms whereas the post-hocs in repeated-measures use un-pooled error terms. In contrast, JASP's defaults result in consistent use of pooled error terms across both types of ANOVA models.

    (As an aside, I recall that for many decades SPSS used pooled error terms for repeated-measures. So they must have changed it recently.)

    R

  • edited April 2023

    @andersony3k

    That is a valid point. However, the attached article advice to refrain from pooled error terms in RM designs.

    But maybe this article is referring to the omnibus test (have only skimmed it through).

    Per😎

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games