Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Scale for the interaction in an ANOVA model

Hello everyone,

I'm a bit confused about the rscaleFixed value that is applied to an interaction. The interaction is coded as the product of two main effects, with the latter coded as sqrt(2)/2 implying the former is coded as 1/2. Does that mean that, for a fixed rscaleFixed value entered into the BF package, there are different scales applied to the main effects and the interactions?

I know that this isn't the case when a whichModels is set to "bottom" because the BF package is coded to default to a ttest in this case (so the difference in coding doesn't matter). But what if whichModels is set to "top"?


  • edited May 2018

    I figured this out. The answer is "yes", different scales are applied to the main effects and the interactions.

    For clarity, I want to mention that my original post did not provide a complete description of the setting that I was asking about. I was referring to the case of a 2x2 fixed effects ANOVA. As it turns out, the difference in coding is more obvious when you do a 2x2x2x2 fixed effects ANOVA.

    I simulated data with the constraint that the sums of squares for ALL main effects and interactions is zero (each cell has exactly the same distribution). Here's the aov() summary:

    Df Sum Sq Mean Sq F value Pr(>F)
    F1 1 0.00 0.000 0 1
    F2 1 0.00 0.000 0 1
    F3 1 0.00 0.000 0 1
    F4 1 0.00 0.000 0 1
    F1:F2 1 0.00 0.000 0 1
    F1:F3 1 0.00 0.000 0 1
    F2:F3 1 0.00 0.000 0 1
    F1:F4 1 0.00 0.000 0 1
    F2:F4 1 0.00 0.000 0 1
    F3:F4 1 0.00 0.000 0 1
    F1:F2:F3 1 0.00 0.000 0 1
    F1:F2:F4 1 0.00 0.000 0 1
    F1:F3:F4 1 0.00 0.000 0 1
    F2:F3:F4 1 0.00 0.000 0 1
    F1:F2:F3:F4 1 0.00 0.000 0 1
    Residuals 144 75.46 0.524

    So there's no variance explained by any of the terms. Here's the BF outputs for with whichModels set to "top":

    Bayes factor top-down analysis

    When effect is omitted from F1 + F2 + F3 + F4 + F1:F2 + F1:F3 + F2:F3 + F1:F4 + F2:F4 + F3:F4 + F1:F2:F3 + F1:F2:F4 + F1:F3:F4 + F2:F3:F4 + F1:F2:F3:F4 , BF is...

    [1] Omit F1:F2:F3:F4 : 2.488178 ±4.05%
    [2] Omit F2:F3:F4 : 4.447647 ±24.45%
    [3] Omit F1:F3:F4 : 3.250185 ±5.16%
    [4] Omit F1:F2:F4 : 3.353204 ±5.62%
    [5] Omit F1:F2:F3 : 3.305366 ±4.36%
    [6] Omit F3:F4 : 4.74362 ±3.53%
    [7] Omit F2:F4 : 4.696784 ±3.53%
    [8] Omit F1:F4 : 4.419148 ±5.93%
    [9] Omit F2:F3 : 4.721122 ±3.53%
    [10] Omit F1:F3 : 4.344021 ±4.76%
    [11] Omit F1:F2 : 4.305297 ±4.33%
    [12] Omit F4 : 5.551244 ±3.75%
    [13] Omit F3 : 6.582877 ±11.44%
    [14] Omit F2 : 6.392093 ±3.53%
    [15] Omit F1 : 6.382501 ±3.52%

    Against denominator:

    DV ~ F1 + F2 + F3 + F4 + F1:F2 + F1:F3 + F2:F3 + F1:F4 + F2:F4 + F3:F4 + F1:F2:F3 + F1:F2:F4 + F1:F3:F4 + F2:F3:F4 + F1:F2:F3:F4

    Bayes factor type: BFlinearModel, JZS

    The error %s are quite high so there's some noise, but it's pretty clear that there are different Bayes Factors for the main effects, the two-way interactions, the three-way interactions, and the four-way interaction. This is because there are different parameterizations for the different interactions (they are coded as the product of the main effects involved; this can be verified by using model.matrix()). Also, a careful reading of Rouder, Morey, Speckman, and Province (2012) section 6 suggests that no adjustment is made for the difference in scales across the main effects, two-way interaction, three-way interactions, ...., and n-way interactions.

    This strikes me as an odd choice. On the one hand, it's kind of consistent with the sparsity-of-effects principle: interactions are in some sense automatically assumed to be of a lower magnitude.

    However, there's some strange features. For example, my intuition is that as the number of factors grows, the evidence for the nth-way interaction also grows such that a trivial n-way interaction can be supported by the data despite a traditional ANOVA showing that it accounts for very little of the variance. The reason that this happens is that the nth-way interaction (when there are two levels per factor) is coded as (1/sqrt(2))^n, so increasing n decreases the numbers used to code the interaction. This effectively makes the scale for the Cauchy distribution for the interaction smaller. Hence, more evidence for the alternative.

    I'm pretty sure that these quirks would go away if the lengths of the interaction vectors were normalized to length one, but I haven't finished doing the math.

  • Dear blindreviewer,

    The questions you raise (also in an earlier post) are of course highly relevant. The Rouder et al. paper is not an easy read, and it would be worthwhile to dissect the reasoning and the resulting performance much more than has been done so far. In my lab we are currently debating similar issues regarding the Rouder et al. ANOVA (not the interactions). A lot of this boils down to pretty subtle modeling choices. As far as specific questions on the Rouder et al. paper are concerned, the chances of getting a meaningful answer are highest if you contact Jeff or Richard directly. Richard does not spend all his time on the forum (as I do :-)) but if you send him an Email and receive a meaningful reply, please post it here!


  • Many thanks, E.J. I found a few additional subtle modeling choices (as you call them) that appear to matter greatly under certain conditions. I will contact Jeff or Richard directly about them, and the issue posted above. Should the discussion prove helpful (I'm not simply making silly mistakes), I will ask for their permission to post info here so everyone benefits.

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games