PROJECT 2 - Notes

Varying font size can be set by using STYLE sheets, or by using something like:
which gives:

You need enough samples (i.e., variable values) for each subject, with enough of a change each time, to be able to produce a noticable effect in your performance metrics, and have enough data points to provide credible statistical analysis, but not so many that each subject takes forever to complete the experiment. It's a tradeoff you need to make and explain.

If you use "time" you'll need to find some way to normalize across tasks. i.e., in order to be comparable, every reading task might need to have the same number of words. Using Speed helps with this problem.

In order to test "comprehension" you'll need to administer a simple test after each reading task. If you use exactly the same text for every reading task in your experiment (i.e., for every variation of your variable) then you will get a Learning Effect, and comprehension should improve over time. However, if you use very different texts for each task then you run the risk of that factor causing the change in the performance measure, rather than the one you are changing. i.e., the text can't be the same, or too different. You need to find a way to deal with this problem.

In order to get statistically significant results in an experiment that makes detailed measurements you'd probably need at least 30 subjects. I don't expect you to go to those lengths: but only 3-4 subjects isn't enough. However, if this was a usability test that observes subjects interacting with an interface, using as few as 5 participants should give a reasonable evaluation of the main strengths and weaknesses of the interface.

The use of Simple Linear Regression Analysis may not be appropriate as the relationship may not be linear, but it is used here as a first approximation to more detailed analysis.

dcb at cs wpi edu