Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Looking for some recommendation to write adaptive testing code

Hi all,

I'm writing an adaptive testing experiment, and thanks to the forum discussion, I learned a lot from here!

By the way, I need some recommendations or guidelines as I'm facing some problem in setting up my experiment. My experiment (Adaptive testing) was to evaluate the hearing status of the participants, therefore, we had a different level (for example, Level 1 to Level 6) of words in the experiment list.

(1) My first experiment trials should be started at the middle level, such as Level 3 (from 1 to 5), and my code was written as below. I wonder if anyone can give me some guide?

(2) If the participants have a correct response, then the next trial will be at a harder level, for example: if the participants have a correct response in level 3, then the next trial will be from level 4. Also, my highest level in the list was 6, and if the participants have the correct response in level 6, the next trial will remain in level 6, the same situation at the lowest level. So I wrote the code as below while it was worked in my e-prime, however, it seems like opensesame doesn't understand what I had written. Any recommendation?

Thanks!

Comments

  • Hi,

    Can you provide more information in how far this does not work? The code looks fine to me. Maybe the positioning in the experimental sequence is wrong, but without more detail, it is hard to pinpoint what is happening.


    Eduard

    Buy Me A Coffee

  • Hi Eduard,

    Thanks for your reply. My first trial of the experiment must be the item of middle level, and I have listed all of the trials into a particular level in the Experiment loop (var.item_level). However, I have tried a couple of times, but the first trial of the experiment sequence may not always be the particular item level that I wanted. Secondly, I have tried to insert either correct or incorrect responses; however, the next trial was not the particular level's item. e.g., I have inserted the correct response in the level 3' item, but the upcoming trial was not the item from level 4. It was randomly assigned from other item_level. So I have no clue that what I should do next, while these two settings were quite essential to my research design.

    I wonder these descriptions would add more information for you to know what I'm wanted to convey. Thanks again! I'm looking forward to your reply.

  • edited July 2020

    Make it easier to know. My experiment design procedure would be like :

    My participants will see a fixation dot, and then stimuli sound on (sound_level [which is according to my var.item_level] start from level 3), then participants answer what they heard. If the participants insert the correct answer, then the next stimuli sound would be from level 4, else would be from level 2.

    #After fixation point, then sound on (sound should be starting from level 3[var.item_level])

    soundP.play() #This might be the part that I'm stuck
    target_blank_canvas.show()
    clock.sleep(1000)
    

    The figure I posted was not clear enough, so I re-post my code down here. Sorry for the inconvenience.

    #start from level 3
    var.item_level == 3
    
    #show fixation point to aware the participants
    fixdot_canvas.show()
    clock.sleep(495)
    
    #After fixation point, then sound on (#sound should draw according to var.item_level)
    soundP.play()
    target_blank_canvas.show()
    clock.sleep(1000)
    
    #then allow participant to answer the question
    t0 = target_answer_canvas.show()
    resp = "" #here we store the response
    
    while True:
    #Get a keyboard response
    key, time = kb.get_key()
    
    #if press return, then the loop should be exited.
    if kb.to_chr(key) == "return":
    break
    
    #handle backspace by removing the last character
    if kb.to_chr(key) == "backspace":
    resp = resp[:-1]
    else:
    resp += kb.to_chr(key)
    
    #save response time
    var.response_time = time - t0
    
    #save all the response key
    self.experiment.set("response", resp)
    
    if resp == var.correct_response: 
    var.correct = 1
    else:
    var.correct = 0 
    
    if var.correct == 1: #if correct = 1, then go to harder level
    var.item_level = var.item_level + 1
    else: #if correct = 0, then go to easier level
    var.item_level = var.item_level - 1
    
    if var.item_level == 7: #when reach top level and have correct response, remains in the same level
    var.item_level = 6
    if var.item_level == 0: #when reach lowest level and have incorrect response, remains in the same level.
    var.item_level = 1
    

    Thanks again! I'm looking forward to your reply.

    TZ

  • HI TZ,

    For a starter, your first line var.item_level == 3 only checks whether the current item level is 3. It doesn't do anything in the sequence, if you want to set the item level to 3, you need to use var.item_level = 3.

    However, if you want the current level to be +1/-1 relative to the previous trial, you can't reset it in the beginning of this block of code. Otherwise, everything that you set on the previous trial (the last lines in your script), will be undone by the first lines in the beginning of the next trial. So ,instead you probably want the item to be set only in the first trial, or even before the trial loop starts. Does that make sense?

    Seeing the experimental loops and variables in there would be helpful to know what is going on. If you say that the level is random on every trial and not incremented from the previous trial, I suspect you have the item level set in the experimental loop and on every trial it is pulling it from that table, rather than using the one you computed. Is that possible?

    Eduard

    Buy Me A Coffee

  • edited July 2020

    Hi Eduard,

    I think I know what makes my works stuck! My experiment procedure will be like (1) fixation (2) stimuli sound (word sound) (3) Response (see the figure below).

    According to the script I wrote, the response step will note all of the responses that participants inserted, which is perfect! However, my stimuli sound won't start from the middle level at the beginning (first trail), which is level 3, for example. Should i add an inline_script var.item_level = 3 in before the experiment loop? or Experiment_procedure? Or somewhere else?

    Then, the sound playing script in the prepare and the run page was different, you can see as below.

    #prepare page

    path = pool(var.sound_experiment)
    soundP = sampler(path)
    

    #Run page

    soundP.play() #sound should draw according to var.item_level
    target_blank_canvas.show()
    clock.sleep(1000)
    

    I think I know the reason why my upcoming trial couldn't run the script I wrote, is it because I didn't define the sound stimulus' level? The code was telling it to run the sound besides telling it to retrieve according to the stimuli sound's level. If yes, how do I modify the code I wrote to satisfy my needs? Any recommendations?

    Appreciate your bits of help! 

    TZ

  • In your loop table, every line corresponds to one trial. So every time that you start a new iteration, item_level is set to the value that you see in the loop table. The adaptive character will be lost. You probably have to remove this columns altogether.

    >Should i add an inline_script var.item_level = 3 in before the experiment loop?

    That is what you should do I think. So after the start_sketchpad and before the ExperimentLoop. Then, the procedure will always start with the level set to 3, and during the loop it will dynamically be adapted depending on the performance (provided that you take out the column).

    If the variable item level should have an effect on a stimulus you'll have to set it of course. For example, if the loudness of the sound depends on the item level, you need to program it, otherwise, you won't see (or hear) any level-specific differences.

    Hope this clears things up.


    Eudard

    Buy Me A Coffee

  • Hi Eduard,

    Thanks for your advice! My experiment was nearly done!

    Now I have some couple question, which are

    (1) I wanted to show the responses of the participants have inserted; however, I have no idea how to write the code after the target_answer_canvas. Any recommendation?

    #show fixation point to aware the participants
    fixdot_canvas.show()
    clock.sleep(495)
    
    #After fixation point, then sound on (#start from level 3)
    soundP.play()
    target_blank_canvas.show()
    clock.sleep(1000)
    
    #then allow participant to answer the question
    t0 = target_answer_canvas.show()
    resp = "" #here we store the response
    while True:
    #Get a keyboard response
    key, time = kb.get_key()
    #if press return, then the loop should be exited.
    if kb.to_chr(key) == "return":
    break
    #handle backspace by removing the last character
    if kb.to_chr(key) == "backspace":
    resp = resp[:-1]
    else:
    resp += kb.to_chr(key)
    

    (2) As the figure below, my stimuli were repeatedly drawn after the break. For example, when my participants heard the sound (stimuli A) from the list of level 1. If he/she answered wrongly, then he/she will still in the level 1 loop; however, there will be a chance to draw exactly the same stimuli A from the previous one.  What modification do I need to do to solve this situation, because I want my ExpLoop not to draw the item/stimuli that have been used in the previous trials?

    Appreciate your bits of help! 

    TZ

  • Hi,

    I have not tested the code, but does this make sense? Basically include the drawing and showing a canvas inside the while loop.

    target_answer_canvas = Canvas()
    resp = "" #here we store the response
    target_answer_canvas.text = Text(resp) 
    t0 = clock.time()
    while True:
        target_answer_canvas.show()
        #Get a keyboard response
        key, time = kb.get_key()
        #if press return, then the loop should be exited.
        if kb.to_chr(key) == "return":
            break
        #handle backspace by removing the last character
        elif kb.to_chr(key) == "backspace":
            resp = resp[:-1]
        else:
            resp += kb.to_chr(key)
            target_answer_canvas.text += kb.to_chr(key)
    

    As for your other question, I unfortunately don't understand the structure of the experiment, the desired behaviour of it, and the problem that you are facing now. Could you elaborate?

    Thanks,

    Eduard

    Buy Me A Coffee

  • edited September 2020

    Hi Eduard,

    Sorry for the late reply! I have tried the code, however, there was an error occur by showing unsupported operand type(s) for +=: 'RichText' and 'str'.

    target_answer_canvas.text += kb.to_chr(key)
    

    The purpose of the present study is to use adaptive testing to shorten the present hearing assessment tasks. We have done some previous studies to ensure that the difficulties of items and the clinical cut-off value. Now, we wanted to examine whether this kind of adaptive assessment tool has any differences with the traditional assessment, so we need to do some pilot testing. The procedure was shown as below.

    However, we face some difficulties while establishing the pilot experiment.

    The most important issue that must be solved was the chance of repeating the same sound stimuli. For example, when the first trial item was 'Apes' from level 3, no matter participants inserted correct or incorrect response, if the following stimuli were drawn from level 3, I don't want to run 'Apes' again. However, in the present setting, the same stimulus may have a chance to show more than 1 time during the experimental procedures. Any recommendations? I have been stuck for a couple of months, it kills me!

    Code:

    Prepare

    #creates fixation dot canvas
    fixdot_canvas = canvas()
    fixdot_canvas.fixdot()
    
    #create target_answer canvas
    target_answer_canvas = canvas(font_size = 50)
    #target_answer_canvas.text('請輸入您的答案', x = 0 , y = 0)
    
    #create target_blank canvas
    target_blank_canvas = canvas(color = 'black')
    target_blank_canvas.fixdot()
    
    #create keyboard response
    kb = keyboard()
    
    #create sound
    path = pool[var.sound_experiment]
    soundP = sampler(path)
    

    Run

    #show fixation point to aware the participants
    fixdot_canvas.show()
    clock.sleep(495)
    
    #After fixation point, then sound on (#start from level 3)
    soundP.play()
    target_blank_canvas.show()
    clock.sleep(1000)
    
    #then allow participant to answer the question
    target_answer_canvas = Canvas()
    resp = "" #here we store the response
    target_answer_canvas.text = Text(resp)
    t0 = clock.time()
    while True:
    target_answer_canvas.show()
    #Get a keyboard response
    key, time = kb.get_key()
    #if press return, then the loop should be exited.
    if kb.to_chr(key) == "return":
    break
    #handle backspace by removing the last character
    elif kb.to_chr(key) == "backspace":
    resp = resp[:-1]
    else:
    resp += kb.to_chr(key)
    target_answer_canvas.text += kb.to_chr(key)
    
    #save response time
    var.response_time = time - t0
    
    #save all the response key
    self.experiment.set("response", resp)
    
    #correct response
    if resp == var.correct_response:
    var.correct = 1
    else:
    var.correct = 0
    
    #switching block by following the rule
    if var.correct == 1:
    var.item_level = var.item_level + 1
    else:
    var.item_level = var.item_level - 1
    
    if var.item_level == 7: # reached highest level
    var.item_level = 6
    
    if var.item_level == 0: #reached lowest level
    var.item_level = 1
    

    Appreciate your bits of help! 

    TZ

  • edited September 2020

    Hi all,

    It's great to come here again! thank to @eduard for your kindly responses! I had solved the question recently.

    However, I'm still have a small question that why I inserted correct answer, but still count as incorrect? For example, the correct response of the trial is 103, but when I insert 103, it doesn't count that the response as correct. See the figure below. What can I do now?


    At last, appreciate for the bit of helps.

    TZ

  • Hi all,

    I solved the question by writing in the inline script.

    if var.response == 103:
       var.correct = 1
    

    Thanks.

    TZ

  • Hi TZ,

    Great that you figured it out by yourself! (and sorry for not getting back to you earlier)

    Eduard

    Buy Me A Coffee

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games