[Solved] Displaying Accuracy and Response Time in Feedback

Berndolino1

Hi,

I'm currently working on a lexical decision task comprising word and non-word stimuli. My problem is that only the average reaction time and accuracy is being displayed in the feedback after having run an experimental loop . How can I see and show the respective average RT and Acc for word and non-word selection? Thanks in advance.

Berndolino

Fab

Hi @Berndolino1,

In a nutshell, the answer to your question is: you must calculate it yourself using code.

You need to use code to manually calculate the % of accuracy and mean RT per condition.

You could do that in Python or in Javascript.

The idea is to increment separate variables to count the word and non-words trials as the block runs, separate variables to count the number of correct responses, and separate variables to add up the RTs for the correct responses in these two conditions.

For example, to count the number of trials in the word condition, the number of trials with a correct response in the word condition and add up the corresponding RTs, you could use the following code in Javascript:

if (vars.condition=="word") {
vars.number_trialst_word= vars.number_trials_word + 1
}

if (vars.condition=="word" && vars.correct==1) {
vars.number_correct_word= vars.number_correct_word + 1;
vars.sumRT_word=vars.sumRT_word + vars.response_time
}

Note that if you want to calculate the mean RT regardless of whether the response was correct or not, you would need to implement the RT sum in the if statement that does not contain the condition of vars.correct==1:

if (vars.condition=="word") {
vars.number_trialst_word= vars.number_trials_word + 1;
vars.sumRT_word=vars.sumRT_word + vars.response_time
}

if (vars.condition=="word" && vars.correct==1) {
vars.number_correct_word= vars.number_correct_word + 1;
}

You'd implement a similar but modified code for the nonword condition.

Then, once the loop has been executed, before displaying the feedback, you would use these variables to calculate the % accuracy ((number of correct responses in specific condition * 100)/number of trials in that condition). For the mean RT, you'd divide the sum of RTs and divide it by he number of correct responses in that condition. Note that you may want to round up these numbers (e.g., in Javascript, using Math.round(your_number)).

Once you have calculated these and attributed them to variables (vars), you'll be able to display the information you want on the feedback screen: "Your % accuracy in the word condition is [accword] and your mean response time is [meanRTword]. Your % accuracy in the non-word condition is [accnonword] and your mean response time is [meanRTnonword]". (I use these variables names as example, they would be whatever you decide them to be when you define them in the code).

Importantly, you must remember to reset all the relevant variables before a block begins, otherwise, if you have several blocks, the feedback your participants get will correspond to performance from the beginning of the task, not just for the latest block. This resetting must be using code too (the feedback reset object in OS won't do, because that object knows nothing of the variables you implemented yourself).

I hope this helps. Have a go and let me know if it is helpful. I'm a bit short on time at the moment, but if I get a chance later, I'll try to put together a basic OS task as an example.

Good luck,

Fabrice.

_____________________________________________________________________________________

If you found my reply helpful and wish to invite me to a coffee, you can do so here 😉: https://www.buymeacoffee.com/psyfab

Berndolino1

Thank you for your quick response. Where exactly can I implement the code in OpenSesame or does it only work using Javascript. If it only works with Javascript how can I use the code for OpenSesame with my own word stimuli?

Fab

Hi @Berndolino1,

The code is to be introduced like any other code in OS. You can either use Python code, using the inline_script object:

Or use Javascript code, using the inline_javascript object:

You'd have to insert code in different places:

• set your counters to zero before the loop containing your trials
• increment the number of trials, correct responses and sum of RT for each of the two conditions (word, nonword) inside the trial sequence, after collecting the subject's response
• calculate the %of correct responses and mean RT for the two conditions after the loop and before displaying the feedback for the block.

Hope this helps.

Fabrice.

Fab

Hi @Berndolino1,

I took a moment to implement an example. Hopefully with this you should be able to adapt your task.

I created a very basic lexical decision task (link to download at the end of this post) in which 6 stimuli are presented (3 words, 3 nonwords). Subjects press one key if the stimulus is a word, and another if it is a non-word (if you prefer to use a "word? yes-no" system, you'd have to adapt the code accordingly).

I used Javascript code instead of Python. You could perfectly rewrite the code in Python language if you wished, it's fairly basic.

Note that I introduced code to make sure that the keys are counterbalanced across subjects based on the subject number (I assume you had already implemented that part but I introduced it just in case - sometimes beginners create two versions of the task to achieve that, and that's really not optimal; introduces more risks of errors as you have two tasks to develop and test etc.). This code some in at the very beginning of the task:

// Code to make sure that the response keys are counterbalanced across subjects based on the subject number
// odd numbered subjects press Z for words and M for nonwords
// even numbered subjects press M for words and Z for nonwords
if (vars.subject_parity == "odd") {
vars.word_response = "z";
vars.nonword_response = "m"
} else {
vars.word_response = "m";
vars.nonword_response = "z"
}

This stores the correct response separately for each condition, based on the subject number. Then, the correct response is set to these variables in the loop:

If you're not experienced with coding in OS, it's worth pointing out the difference between "var" and "vars".

var text_variable= "hello"

... creates a variable called text_variable that contains the string "hello", but this variable is only available in the inline_javascript object in which it is defined (so, if you tried to use this variable later in the task, it's content would be empty).

vars.text_variable = "hello"

... also creates a variable called text_variable that contains the string "hello", but this variable is accessible throughout the task. So in the example we're interested in, where you want trials counters and sums of RTs that are used across several trials and feedback objects, we need to use "vars".

Here's the general structure of the task:

As you can see, I have introduced two types of feedback: feedback per trial and feedback per block. As the task runs, I also output information to the console so that you can see in real time what's going on.

Now to the Javascript code...

Step 1: initialize counters (this bit must run before each block, if you have several blocks)

Step 2: Incrementing the counters in the trial sequence (after the response is collected, obviously)

Code:

// counts number of trial, number of trials with correct response and sum of RT for the WORD condition
// note that in this script, the sum of RTs is calculated on all trials from this condition: correct and incorrect
if (vars.condition == "word") {
vars.number_trials_word= vars.number_trials_word + 1;
vars.sumRT_word=vars.sumRT_word + vars.response_time
}
if (vars.condition == "word" && vars.correct==1) {
vars.number_correct_word= vars.number_correct_word + 1;
}

// counts number of trial, number of trials with correct response and sum of RT for the WORD condition
// note that in this script, the sum of RTs is calculated on all trials from this condition: correct and incorrect
if (vars.condition == "nonword") {
vars.number_trials_nonword= vars.number_trials_nonword + 1;
vars.sumRT_nonword=vars.sumRT_nonword + vars.response_time
}
if (vars.condition == "nonword" && vars.correct==1) {
vars.number_correct_nonword= vars.number_correct_nonword + 1;
}

// To check it is working, outputs to the console the block-wise counters being incremented
console.log("_____________________TRIAL DATA___________________________")
console.log("Condition: "+vars.condition)
console.log("Target: "+vars.target+", Response: "+vars.response+", Correct response: "+vars.correct_response+", RT: "+vars.response_time+" ms")
console.log("_____________________BLOCK DATA___________________________")
console.log ("Condition: "+vars.condition+", "+ "Response: "+vars.response)
console.log ("Number of word trials: "+vars.number_trials_word)
console.log ("Number of word trials with correct response: "+vars.number_correct_word)
console.log ("Sum of RTs (word): "+vars.sumRT_word)
console.log ("Number of nonword trials: "+vars.number_trials_nonword)
console.log ("Number of nonword trials with correct response: "+vars.number_correct_nonword)
console.log ("Sum of RTs (nonword): "+vars.sumRT_nonword)

Step 3: compute the block-wise feedback (outside the loop, right after it)

// calculates % of correct responses in each condition
vars.pcword=100*vars.number_correct_word/vars.number_trials_word
vars.pcnonword=100*vars.number_correct_nonword/vars.number_trials_nonword

// calculates mean RT in each condition (for all trials from that condition, correct and incorrect)
// (If you want to calculate it for correct responses only, you'll have to modify the calculation of the sum of RTs in "increment_counters"
// and modify the code below to use the numbner of trials with correct response as the denumerator)
vars.meanRTword=vars.sumRT_word/vars.number_trials_word
vars.meanRTnonword=vars.sumRT_nonword/vars.number_trials_nonword

Then, you can use a feedback object and insert text to display the information you want (based on the variables defined earlier):

Et voilá!

Take the time to go through the code. I inserted notes throughout, so it'll be easy to follow. Once you've grasped the key aspects of it, no doubt you'll be able to implement it in your own task. I recommend using the console to output useful information as you develop the task; it makes it easier to spot possible problems. If you're not familiar with Javascript, I recommend searching the web for solutions. I started using Open Sesame in March and had no knowledge of Javascript, but I found loads of useful pieces of code out there (it's easier to find code and adapt it than to start from scratch, especially if you're new to a programming language). Same applied to Python (though in my case, so far I've stuck to Javascript because Python code is not supported in OSWeb; so if you plan to run your task online, better use Javascript).

Sometimes, there are ways to achieve things without any coding at all (see this thread for an example: https://forum.cogsci.nl/discussion/comment/23736#Comment_23736), but in your case, in order to display feedback per condition, there was no other way (OS's default feedback options are calculated o all trials of a block, there is no way of differentiating between conditions).

Here is the task I programmed for this example:

Feedback per condition.osexp

Good luck!

Fabrice.

_____________________________________________________________________________________

If you found my reply helpful and wish to invite me to a coffee, you can do so here 😉: https://www.buymeacoffee.com/psyfab

Fab

PS: when replying in the forum, best practice is to use the correspondent's handle, so that they get notified of your reply. To do so, just type "@" and the beginning of your correspondent's handle. For example, if you type "@Fa", you 'll see this appear: