Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

BF between 1 and 3, cannot collect more data, meaningful to report ?

I have a question about the interpretation of anecdotal/inconclusive evidence.
My sample size is quite big given the standards in my field (N=75), and I cannot collect extra data. Bayesian t-tests for my contrasts of interest show BFs between 1 and 3, suggesting inconclusive evidence. Some of these go together with p-values around 0.05 in classic t-test (in the unexpected direction).
Can I report/interpret such results in a meaningful way? E.g., can I state "if there is an effect in the population, it is not strong or consistent enough to be detected with this sample"?
I am asking for practical reasons. If given these outcomes, I would absolutely have to collect more data to draw any meaningful conclusions, I cannot report the experiment. Because I cannot collect more data. It would be a pitty, because the conclusion that there is not enough evidence to show an effect, would be in line with two other experiments that showed support for the nullhypothesis and are to go into the same publication.

Comments

  • Hi Hanna,

    Here's my take on this: Inconclusive evidence is a perfectly valid outcome of an experiment, even though perhaps not very satisfying; sometimes life is such that you try to find out something but don't succeed. There is certainly nothing wrong with reporting that.

    Whether it is meaningful, in the sense of theoretically interesting, depends on the context. For example, if you're trying to replicate a paper that showed a huge effect with N=10, then inconclusive evidence with N=75 would suggest that the initial paper at least overestimated the effect—that might be worthwhile. But in other cases it may not be. The interest, or lack thereof, of your result is not primarily a statistical question. How do you feel about it?

    Cheers,
    Sebastiaan

  • Hi Hanna,

    This is a great question. Some brief thoughts:
    1. With BFs in between 1 and 3 you have not learned much with respect to the specific model-comparison question you were studying. with N=75, the posterior distribution will be relatively peaked -- so you did learn a lot about the size of the effect (as Sebastiaan also points out). You may also have learned a lot about whether the effect is positive or negative.
    2. You mention that the effect goes in the opposite direction. Was your Bayesian test one-sided? If the effect goes in the opposite direction and is almost significant, I would expect pretty good evidence in favor of the null. This depends of course on how reasonable it is to employ the order-restriction. If the theory you are testing clearly stipulates the order-restriction, then your prior ought to reflect that, imo.

    Cheers,
    E.J.

  • Dear both,
    Thank you both for your responses. A lot happened last year, and I must have lost track of this thread. But I wanted to give a shout-out to you to thank you for taking the time to answer questions here! Much appreciated and very helpful! I'll try to find out what happened with this data to let you know. I think we reported it as inconclusive evidence.
    Hanna

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games