Looking for some recommendation to write adaptive testing code

Tzlim

Hi all,

I'm writing an adaptive testing experiment, and thanks to the forum discussion, I learned a lot from here!

By the way, I need some recommendations or guidelines as I'm facing some problem in setting up my experiment. My experiment (Adaptive testing) was to evaluate the hearing status of the participants, therefore, we had a different level (for example, Level 1 to Level 6) of words in the experiment list.

(1) My first experiment trials should be started at the middle level, such as Level 3 (from 1 to 5), and my code was written as below. I wonder if anyone can give me some guide?

(2) If the participants have a correct response, then the next trial will be at a harder level, for example: if the participants have a correct response in level 3, then the next trial will be from level 4. Also, my highest level in the list was 6, and if the participants have the correct response in level 6, the next trial will remain in level 6, the same situation at the lowest level. So I wrote the code as below while it was worked in my e-prime, however, it seems like opensesame doesn't understand what I had written. Any recommendation?

Thanks!

eduard

Hi,

Can you provide more information in how far this does not work? The code looks fine to me. Maybe the positioning in the experimental sequence is wrong, but without more detail, it is hard to pinpoint what is happening.

Eduard

Tzlim

Hi Eduard,

Thanks for your reply. My first trial of the experiment must be the item of middle level, and I have listed all of the trials into a particular level in the Experiment loop (var.item_level). However, I have tried a couple of times, but the first trial of the experiment sequence may not always be the particular item level that I wanted. Secondly, I have tried to insert either correct or incorrect responses; however, the next trial was not the particular level's item. e.g., I have inserted the correct response in the level 3' item, but the upcoming trial was not the item from level 4. It was randomly assigned from other item_level. So I have no clue that what I should do next, while these two settings were quite essential to my research design.

I wonder these descriptions would add more information for you to know what I'm wanted to convey. Thanks again! I'm looking forward to your reply.

Tzlim

Make it easier to know. My experiment design procedure would be like :

My participants will see a fixation dot, and then stimuli sound on (sound_level [which is according to my var.item_level] start from level 3), then participants answer what they heard. If the participants insert the correct answer, then the next stimuli sound would be from level 4, else would be from level 2.

#After fixation point, then sound on (sound should be starting from level 3[var.item_level])

soundP.play() #This might be the part that I'm stuck
target_blank_canvas.show()
clock.sleep(1000)

The figure I posted was not clear enough, so I re-post my code down here. Sorry for the inconvenience.

#start from level 3
var.item_level == 3

#show fixation point to aware the participants
fixdot_canvas.show()
clock.sleep(495)

#After fixation point, then sound on (#sound should draw according to var.item_level)
soundP.play()
target_blank_canvas.show()
clock.sleep(1000)

#then allow participant to answer the question
t0 = target_answer_canvas.show()
resp = "" #here we store the response

while True:
#Get a keyboard response
key, time = kb.get_key()

#if press return, then the loop should be exited.
if kb.to_chr(key) == "return":
break

#handle backspace by removing the last character
if kb.to_chr(key) == "backspace":
resp = resp[:-1]
else:
resp += kb.to_chr(key)

#save response time
var.response_time = time - t0

#save all the response key
self.experiment.set("response", resp)

if resp == var.correct_response: 
var.correct = 1
else:
var.correct = 0 

if var.correct == 1: #if correct = 1, then go to harder level
var.item_level = var.item_level + 1
else: #if correct = 0, then go to easier level
var.item_level = var.item_level - 1

if var.item_level == 7: #when reach top level and have correct response, remains in the same level
var.item_level = 6
if var.item_level == 0: #when reach lowest level and have incorrect response, remains in the same level.
var.item_level = 1

Thanks again! I'm looking forward to your reply.

TZ

eduard

HI TZ,

For a starter, your first line var.item_level == 3 only checks whether the current item level is 3. It doesn't do anything in the sequence, if you want to set the item level to 3, you need to use var.item_level = 3.

However, if you want the current level to be +1/-1 relative to the previous trial, you can't reset it in the beginning of this block of code. Otherwise, everything that you set on the previous trial (the last lines in your script), will be undone by the first lines in the beginning of the next trial. So ,instead you probably want the item to be set only in the first trial, or even before the trial loop starts. Does that make sense?

Seeing the experimental loops and variables in there would be helpful to know what is going on. If you say that the level is random on every trial and not incremented from the previous trial, I suspect you have the item level set in the experimental loop and on every trial it is pulling it from that table, rather than using the one you computed. Is that possible?

Eduard

Tzlim

Hi Eduard,

I think I know what makes my works stuck! My experiment procedure will be like (1) fixation (2) stimuli sound (word sound) (3) Response (see the figure below).

According to the script I wrote, the response step will note all of the responses that participants inserted, which is perfect! However, my stimuli sound won't start from the middle level at the beginning (first trail), which is level 3, for example. Should i add an inline_script var.item_level = 3 in before the experiment loop? or Experiment_procedure? Or somewhere else?

Then, the sound playing script in the prepare and the run page was different, you can see as below.

#prepare page

path = pool(var.sound_experiment)
soundP = sampler(path)

#Run page

soundP.play() #sound should draw according to var.item_level
target_blank_canvas.show()
clock.sleep(1000)

I think I know the reason why my upcoming trial couldn't run the script I wrote, is it because I didn't define the sound stimulus' level? The code was telling it to run the sound besides telling it to retrieve according to the stimuli sound's level. If yes, how do I modify the code I wrote to satisfy my needs? Any recommendations?

Appreciate your bits of help! 

TZ

eduard

In your loop table, every line corresponds to one trial. So every time that you start a new iteration, item_level is set to the value that you see in the loop table. The adaptive character will be lost. You probably have to remove this columns altogether.

>Should i add an inline_script var.item_level = 3 in before the experiment loop?

That is what you should do I think. So after the start_sketchpad and before the ExperimentLoop. Then, the procedure will always start with the level set to 3, and during the loop it will dynamically be adapted depending on the performance (provided that you take out the column).

If the variable item level should have an effect on a stimulus you'll have to set it of course. For example, if the loudness of the sound depends on the item level, you need to program it, otherwise, you won't see (or hear) any level-specific differences.

Hope this clears things up.

Eudard

Tzlim

Hi Eduard,

Thanks for your advice! My experiment was nearly done!

Now I have some couple question, which are

(1) I wanted to show the responses of the participants have inserted; however, I have no idea how to write the code after the target_answer_canvas. Any recommendation?

#show fixation point to aware the participants
fixdot_canvas.show()
clock.sleep(495)

#After fixation point, then sound on (#start from level 3)
soundP.play()
target_blank_canvas.show()
clock.sleep(1000)

#then allow participant to answer the question
t0 = target_answer_canvas.show()
resp = "" #here we store the response
while True:
#Get a keyboard response
key, time = kb.get_key()
#if press return, then the loop should be exited.
if kb.to_chr(key) == "return":
break
#handle backspace by removing the last character
if kb.to_chr(key) == "backspace":
resp = resp[:-1]
else:
resp += kb.to_chr(key)

(2) As the figure below, my stimuli were repeatedly drawn after the break. For example, when my participants heard the sound (stimuli A) from the list of level 1. If he/she answered wrongly, then he/she will still in the level 1 loop; however, there will be a chance to draw exactly the same stimuli A from the previous one.  What modification do I need to do to solve this situation, because I want my ExpLoop not to draw the item/stimuli that have been used in the previous trials?

Appreciate your bits of help! 

TZ

eduard

Hi,

I have not tested the code, but does this make sense? Basically include the drawing and showing a canvas inside the while loop.

target_answer_canvas = Canvas()
resp = "" #here we store the response
target_answer_canvas.text = Text(resp) 
t0 = clock.time()
while True:
    target_answer_canvas.show()
    #Get a keyboard response
    key, time = kb.get_key()
    #if press return, then the loop should be exited.
    if kb.to_chr(key) == "return":
        break
    #handle backspace by removing the last character
    elif kb.to_chr(key) == "backspace":
        resp = resp[:-1]
    else:
        resp += kb.to_chr(key)
        target_answer_canvas.text += kb.to_chr(key)

As for your other question, I unfortunately don't understand the structure of the experiment, the desired behaviour of it, and the problem that you are facing now. Could you elaborate?

Thanks,

Eduard

Tzlim

Hi Eduard,

Sorry for the late reply! I have tried the code, however, there was an error occur by showing unsupported operand type(s) for +=: 'RichText' and 'str'.

target_answer_canvas.text += kb.to_chr(key)

The purpose of the present study is to use adaptive testing to shorten the present hearing assessment tasks. We have done some previous studies to ensure that the difficulties of items and the clinical cut-off value. Now, we wanted to examine whether this kind of adaptive assessment tool has any differences with the traditional assessment, so we need to do some pilot testing. The procedure was shown as below.

However, we face some difficulties while establishing the pilot experiment.

The most important issue that must be solved was the chance of repeating the same sound stimuli. For example, when the first trial item was 'Apes' from level 3, no matter participants inserted correct or incorrect response, if the following stimuli were drawn from level 3, I don't want to run 'Apes' again. However, in the present setting, the same stimulus may have a chance to show more than 1 time during the experimental procedures. Any recommendations? I have been stuck for a couple of months, it kills me!

Code:

Prepare

#creates fixation dot canvas
fixdot_canvas = canvas()
fixdot_canvas.fixdot()

#create target_answer canvas
target_answer_canvas = canvas(font_size = 50)
#target_answer_canvas.text('請輸入您的答案', x = 0 , y = 0)

#create target_blank canvas
target_blank_canvas = canvas(color = 'black')
target_blank_canvas.fixdot()

#create keyboard response
kb = keyboard()

#create sound
path = pool[var.sound_experiment]
soundP = sampler(path)

Run

#show fixation point to aware the participants
fixdot_canvas.show()
clock.sleep(495)

#After fixation point, then sound on (#start from level 3)
soundP.play()
target_blank_canvas.show()
clock.sleep(1000)

#then allow participant to answer the question
target_answer_canvas = Canvas()
resp = "" #here we store the response
target_answer_canvas.text = Text(resp)
t0 = clock.time()
while True:
target_answer_canvas.show()
#Get a keyboard response
key, time = kb.get_key()
#if press return, then the loop should be exited.
if kb.to_chr(key) == "return":
break
#handle backspace by removing the last character
elif kb.to_chr(key) == "backspace":
resp = resp[:-1]
else:
resp += kb.to_chr(key)
target_answer_canvas.text += kb.to_chr(key)

#save response time
var.response_time = time - t0

#save all the response key
self.experiment.set("response", resp)

#correct response
if resp == var.correct_response:
var.correct = 1
else:
var.correct = 0

#switching block by following the rule
if var.correct == 1:
var.item_level = var.item_level + 1
else:
var.item_level = var.item_level - 1

if var.item_level == 7: # reached highest level
var.item_level = 6

if var.item_level == 0: #reached lowest level
var.item_level = 1

Appreciate your bits of help! 

TZ

Tzlim

Hi all,

It's great to come here again! thank to @eduard for your kindly responses! I had solved the question recently.

However, I'm still have a small question that why I inserted correct answer, but still count as incorrect? For example, the correct response of the trial is 103, but when I insert 103, it doesn't count that the response as correct. See the figure below. What can I do now?

At last, appreciate for the bit of helps.

TZ

Tzlim

Hi all,

I solved the question by writing in the inline script.

if var.response == 103:
   var.correct = 1

Thanks.

TZ

eduard

Hi TZ,

Great that you figured it out by yourself! (and sorry for not getting back to you earlier)

Eduard