2-way-anova with balance over subjects

I have a 2x2 design measuring 50 subjects reactiontimes in congruant and incongruant conditions, using 2 different devices (joystick, responsepad).

My main effect is within devices (congruant trials should be faster than incongruant trials), my interaction effect is between devices(In the joystick-setup the effect should be more pronounced than for the Responsepad). I want to balance over subjects (since some subjects might be naturally slower or faster than others) and over the stimulus set (each subjects sees the same stimuli and I want to account for some stimuli being more difficult to react to).

Each row of my datatable would therefor look roughly like this:

SubjectNr StimulusNr Device Congruant/Incongruant Reactiontime

1 1 1 1 0.9213

1 2 2 1 0.4123

1 3 1 2 0.3123

1 4 2 2 0,5123

2 1 ... ... ...

The first 4 rows are obviously categorical variables.

After reading the anova, ranova and fitrm in Matlab several times I still struggle how to implement my analysis. Im thankfull for any help

Best Jo

Answer 1

[Jeff Miller]

Look here for an explanation of how to do a repeated measures ANOVA with the same 2x2 within-Ss structure.

[jasony]

I do understand how the 2 x 2 there works, but I am not sure how I can additionally balance over the stimulus set.

In the example you point out, there is one line in the datatable per subject - I have roughtly 200 lines per subject (since each subject has seen 200 stimuli in different conditions).

I am not sure how to transfere this

[Jeff Miller]

You said you had a 2 x 2 design, and I understand that in terms of Device x Con/Inc. So I thought the StimulusNr column in your original table was not needed (just a combination indicating the four cells of your 2 x 2 design).

I do not understand this additional "stimulus set" factor--sorry. How do the 200 different stimuli differ from one another? Are there are 50 different stimuli in each of the 4 conditions of this 2 x 2? Or maybe 100 different Con stimuli and 100 different Inc stimuli, each presented with both devices?

To put it another way: Suppose that for each subject you simply compute the 4 averages for the 2 x 2 conditions, averaging across stimulus set, and then perform ANOVA on that table of 50 (subjects) x 4 (conditions) numbers. What information do you want that this analysis would not provide?

[jasony]

I very much appreciate your engagement :)

I have 100 different pictures (the stimuli).

The subjects go through 4 blocks that differ in instruction (congruant / incongruant) and device (button press / joystick.)

For convienice I only choose those block orders, that make the subjects switch device once - so each subjects get one of the following block orders (J = Joystick, B = Button-press C = congruant I = Incongruant):

JC JI BC BI

JI JC BI BC

BC BI JC JI

BI BC JI JC

Each stimulus is shown once over the first two blocks and once over the second two blocks (so I initially have 200 datapoints per subject). This means every subject is seeing every simulus two times (once per device) but not necessarily once per instruction (it might see the stimulus once in JC and once in BC - so in both congruant blocks). For now I do not want to account for stimulus order or for the subjects have seen the stimulus already or not.

Additionally, wrong trials (so if the subjects did not follow the instruction correctly) are excludet from analysis, therefor I do not end up with quite 200 datapoint per subject, but between 180-195 (dependent on how accurate the subject performed).

Now for some stimuli, it might be significantly harder to follow the instruction than for others, hence it might very well be that the reaction time does not only depend on the device and the instruction, but also on the specific stimulus.

I want to counterbalance for this effect. (I am not interested in the effect of the stimuli, but I dont want it to scer my data).

Thank you so much for your efford already :)

Best Johannes

[Jeff Miller]

That is clear--thanks for the explanation.

I assume that the 100 different pictures are chosen pseudo-randomly to represent some larger set of possible pictures (much like the subjects represent possible people). In that case, your experiment formally has two random elements--subjects and pictures--and you would need linear mixed models to accommodate both of these "random factors". The MATLAB command for this is 'fitlme', but I don't have a lot of experience with that and so cannot say what the model command would be.

I also assume that the pictures were assigned randomly to congruent/incongruent conditions within a device. If you are not interested in the effect of stimuli, I think you should just trust that the random assignment appropriately balanced the stimuli across conditions (so stimulus differences would not skew your results), and analyze the 4 condition averages per subject.

A final comment: from experience with this sort of experiment, my impression is that there will actually be large order effects unless the subjects already have a lot of practice with the task. So, I would not ignore these order effects (doing so may reduce your power to detect the device and congruence effects). The simplest way to include order effects is to have a between-subjects order factor with three levels.

[jasony]

Thank you very much for your help.

True, I think I have to use linear mixed models here - I think this is also why I didnt get any further with the ANOVA at this point. I gonna do it like you say and for now average over the 4 conditions within the subjects and do a 2x2 ANOVA.

Than I gonna go for a second step and try it with linear mixed models. That all helped me a lot, thank you very much for your effort, I realy apprciate it :)

Best

Johannes

[Jeff Miller]

You are welcome. Good luck with your research.

If you get around to looking at delta plots (as some people do with congruence effects), you might find some of the routines in RawRT useful.