Research Methods II: Autumn Term 2004

Writing research reports: A brief summary (to supplement "Introductory Guidelines on Lab Course Work" which you received in the first year).

This is a very brief guide to the sections your report should contain, and what to put in them. 

Some points on writing style
In general, past tense should be used though out in describing results and procedures from your study and other studies.
Whenever you draw a comparison ALWAYS state what is being compared with what. For example, do not simply say “males performed better on science fiction”.  Do you mean “than romance” or “than females”? One can be true without the other being true, so you have to say explicitly what the comparison is.

Abstract
A brief (half-page) summary of your experiment, including the main aims, what you did, and the main findings.  When planning your abstract, answer these questions as concisely as possible:
What was the aim of the study? What method(s) were used? What were the major findings?
What were concluded from these findings? Only give a conceptual overview, do not give details like numbers of subjects run, p values, type of analysis, or fine details of method or design.

Introduction
If you know of some relevant background literature, briefly outline the previous findings, and say how your experiment derives from them/develops them/relates to them more generally.  Explain the purpose of your experiment, say why you included the variables you did, and what you expect to find (perhaps based on previous work, or on your own views about how the variables you have included might affect people's performance).  These are your predictions.  In this module, you might or might not have specific predictions about the main effects of your two variables on performance, but you should definitely have predictions about how you expect the variables to interact, and why.

Here are some questions to ask yourself as you plan this section:
What was the purpose of my study?
How does my study build on or derive from other studies? (but note that for this course you are NOT expected to do a literature review, if you have come across something that's relevant, mention it.  If not, don't worry.  Just explain clearly why YOU think it's interesting).
What was my working hypothesis or expectation?

Method
Usually, the method section will have four sub-sections: subjects, materials, design and procedure.  Sometimes there is a separate section for "apparatus" or "equipment", but since you're probably not using anything that needs a technical description, you can omit this.

Subjects
Say how many subjects were in your experiment, how they were allocated to conditions, how many were in each condition and how they were recruited (NOT randomly!). Who were the subjects - students?  If you think it's relevant, you could mention other characteristics: age range, major.
Materials
Explain how you chose the texts for your experiment. Explicitly state your criteria. Give exact sources of texts you use. If you made up, or altered, texts, say how you did this.  How long were the texts?  Do not give length in ambiguous units like “one side”, be more precise.  Anything else the reader needs to know about their content.  Any tests you designed (e.g. multiple choice, sentence recognition) should also be described here. How did you choose sentences or questions? How many were there? What type of answers were required by the questions? On what basis did you construct false sentences? How did you determine key concepts?
Design
Specify dependent and independent variables. What were the names and levels of the IVs?  Which factors were within-subjects (also called repeated measures), and which between-subjects? 
If there were any issues about order of presentation, or counterbalancing of materials (texts) how were these dealt with?
For example, you could say:
“Design. This experiment used a 2 X 3 (sex (males vs females) by text type (romance vs pornography vs car maintenance) mixed design, with repeated measures on the second factor. The order of texts was fully counterbalanced across subjects. The dependent variable was text recall.” (NB: “fully counterbalanced” means all possible orders were used equally often across subjects; a “mixed design” means there both within and between subjects factors.) Make sure you understand these terms and concepts. Last year some people simply copied this paragraph, or parts of it, when it was completely inappropriate for their experiment!
Procedure
Where were the subjects tested?  Individually, or in a group?  What instructions did you give them (if these are lengthy, you could include them in an appendix)?   How long did the experiment take?  How did you test the subjects? Were they debriefed?

Results
First, you may need to describe how you scored your results, and derived scores for each subject.  If there were any problems with scoring, or you needed to get second raters and then agree ratings, describe your procedure.  Then, show a table of your data (mean performance in each condition and standard deviations), and refer to it in the text.  If you display your results graphically, beware of the range of values SPSS uses for each axis in producing a graph. SPSS will choose that range that makes the effect look most impressive. If you have a non-significant interaction, for example, SPSS’s default graph might look like it illustrates a strong interaction. But if you change the y-axis from ranging from 6.2 to 7.5 concepts recalled to 0 to 20 concepts recalled, lines will suddenly look a lot more parallel. Make sure your graph visually supports rather than contradicts your conclusions. Make sure the x-axis has meaningful labels for groups, not numbers.

Then see the SPSS handout for this module for an example results section for a two way ANOVA. You do not need to describe the details of how you interacted with SPSS to derive the ANOVA summary table, or any other results. If your introduction contains predictions for simple effects make sure you test precisely the simple effects you made predictions about.

Discussion
In this final main section, bring together the different sections of your paper.  Review your predictions, from the Introduction, and say whether or not they were confirmed.  Look for any shortcomings in your study, and suggest how they might be remedied in further work.  If you can think of any, suggest further experiments that might extend your own findings.

Do not assert that there are differences that you haven’t tested and found significant. For example, say you have found that males performed significantly better on science fiction rather than romance (simple effect of text type for males) and that females performed significantly better on romance rather than science fiction (simple effect of text type for females). If these are the only simple effects you have tested, you cannot conclude (even if the means are in the right direction) that males were better than females on science fiction - that is the simple effect of gender for science fiction, something you haven’t tested.

Some questions to ask yourself:
What was the aim of my study and what were the main predictions?
How do my results relate to that purpose and predictions?
How valid and generalizable are my findings?
Are there wider implications of these findings that could be explored?

References
List, alphabetically, in standard reference format, any works you referred to in your paper. If you have not read the paper yourself, list it as “cited in...” wherever you saw it referred to.

Appendices
If you think it is useful to the reader, provide some of the material in the appendices: these might include full instructions for the experiment, example passages, examples of scoring procedures, etc. Also include SPSS output, raw data, and histograms.

Tables and Figures
In papers submitted for publication to journals, these are usually included right at the end.  However, it is probably better to include them in the appropriate place in the text of your report, as they would appear in a published journal article.