We have this debate often. How well do college students represent your target population when they are the majority of our research participants? Is the gap worth the savings in time and money needed to recruit them? How generalizable are our findings? Does this introduce fundamental construct validity problems and skew our conclusions one way or the other?
So this research probably won’t surprise anyone, but I think it adds an important piece to the puzzle.
Many studies use multiexperiment designs where experiments are carried out at different times of semester. When comparing between experiments, the data may be confounded by between-participants effects related to motivation. Research indicates that course-credit participants who engage in research early in semester have different personality and performance characteristics compared to those tested late in semester…
What they found is that college students who participate for money have about the same performance whether they are tested at the beginning or at the end of the semester. But for those participating for credit, their intrinsic motivation significantly declines near the end of the semester, their performance declines, and the variability in their performance increases (which masks potentially significant results).
So as researchers we probably want to be very careful with any studies near the end of the semester that use course credit as the compensation. Or intentionally schedule your data collection to be done before students get exhausted studying for midterms. While the students might be tempted to procrastinate, if all of the studies are early they won’t have a choice.