Research Methods and Design I
PSY70310
http://web.jjay.cuny.edu/~mkovera/methods
Mondays, 12:30-2:30
2347N
Instructor: Margaret Bull Kovera, Ph.D.
2118A North Hall
212.484.1112
Office Hours: by appointment
Required Texts
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Silva, P. (2007). How to write a lot: A practical guide to
productive academic writing.
Course Description
In this course, we will cover the fundamentals of research methods and design. The course is designed to teach you the skills required to evaluate the research of others and to design, conduct, and report on research of your own. The scientific process relies on both theory and data to describe, explain, predict, and/or influence some phenomenon of interest. We will be focusing on theory development and research design, with some coverage of data analysis and measurement. We will discuss the logic underlying the research process as well as a broad range of design and analytic methods. Throughout the course there will be an emphasis on both conceptual understanding and the development of practical “how-to” skills. Topics covered in the sequence are organized in terms of the stages of the research process, beginning with theory building and ending with interpretation and verification. We will not cover measurement in depth as it is the topic of a separate course (Psychometrics). Learning methods will include lecture, discussion and application, written assignments, and exams.
Course Requirements
Attendance in class. I
expect that you will attend class every week. If you have to miss a class, it
is your responsibility to get notes from a fellow student.
Participation in class discussion. I expect that you will complete the readings
for each week and will come to class ready to discuss the concepts contained in
the readings. Your participation will be
graded based on its quality and quantity.
You will receive a zero for class participation on any day that you are
absent or fail to participate. There is
no way to makeup for missed participation but I will drop your lowest
participation grade. Class participation
is worth 10% of your grade.
Exams. You will have a mid-term and final exam to
test your comprehension of the material covered in class. The exams will be some combination of
multiple choice, short answer, and essay questions. These exams will each be worth 30% of your
grade (60% total). The mid-term exam
will take place on October 27th
and the final exam will take place on December
22nd.
Ethics
Certification. You must
complete the online training in the ethics involved in research with human
participants. The training can be found
at https://www.citiprogram.org/default.asp. You must turn
in evidence of your completed certification by September 22nd and is worth 5% of your grade.
Article
Presentation. When thinking about research methods, we often focus
on the shortcomings of research questions, research methods, and data
analysis. This focus distracts us from
thinking about the characteristics of research that excites us. For your presentation, I want you to choose
an article that you think employs research methods to which we should all
aspire in our research. Your
presentation should provide a description of the research questions, the
methods used, and your evaluation of why this article represents an example of
high-quality research. Given the number of students in the class, presentations
will be brief by necessity.
Presentations will be held on December
7th and 15th;
they are worth 5% of your
grade.
Article Review.
During the course of the semester, you will learn how to evaluate research
articles, culminating in a lesson on the peer-review process. To demonstrate your integration of the course
material, you will write a 3-5 page single-spaced review of a research article
drawn from a list of articles that I will provide to you. More details about what you are expected to
cover in your review and a list of articles from which you will choose your
article to review will be provided later in the semester. This assignment will be worth 20% of your grade and will be judged on
the accuracy of the review of the research question, hypotheses, and design,
the quality of the critical analysis of the research methods described
(including a discussion of validity threats), and the insightfulness of the
recommendations for improving the manuscript or the research described in the
article. If you are presenting on
December 7th, this assignment is due before class on December 15th. If you are presenting on December 15th,
this assignment is due before class on December
7th. Please submit the
assignment to me via email at mkovera@jjay.cuny.edu.
If you have to miss a class for a religious
holiday/purpose or require special accommodation due to a learning disability,
please let me know during the first week of the semester.
|
Date |
Class |
Topics Covered |
|
|
9/8 |
Introductions |
|
|
|
9/15 |
Philosophy of Science |
Theory development, hypothesis generation and testing |
McGuire, W. J. (2004). A perspectivist approach to theory construction. Personality and Social Psychology Review, 8, 173-182. Serlin, R. C., & Lapsley, D. K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist, 40, 73-83. Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108-141. McGuire, W. J. (1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual Review of Psychology, 48, 1-30. Daft, R.
L., |
|
9/22 |
Research Ethics |
Ethical conflicts, use of deception, studying vulnerable populations, debriefing, IRB regulations |
Baumrind, D. (1964). Some thoughts on ethics of research after reading Milgram's "Behavioral Study of Obedience." American Psychologist, 19, 421-423. Milgram, S. (1964). Issues in the study of obedience: A reply to Baumrind. American Psychologist, 19, 848-852. Hertwig, R., & Ortmann, A (2008). Deception in experiments: Revisiting the arguments in its defense. Ethics & Behavior, 18, 59-92. Smith, S. S., & Richardson, D. (1983). Amelioration of deception and harm in psychological research: The important role of debriefing. Journal of Personality and Social Psychology, 44, 1075-1082. Pasquerella, L. (2002). Confining choices: Should inmates' participation in research be limited? Theoretical Medicine and Bioethics, 23, 519-536. Complete human participants certification at https://www.citiprogram.org/default.asp |
|
10/6 |
Threats to Validity I |
Statistical conclusion validity, internal validity, experimenter expectancy effects, confounds |
SCC, Chapters 1- 2 Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-387. Rosenthal, R. (2002). Covert communications in classrooms, clinics, courtrooms, and cubicles. American Psychologist, 57, 839-849. Larzelere, R. E., Kuhn, B. R., & Johnson, B. (2004). The intervention selection bias: An underrecognized confound in intervention research. Psychological Bulletin, 130, 289-303. McCleary,
R., & Meeker, J. W. (2006). Do peep shows
'cause' crime? A response to |
|
10/14 Tuesday |
Threats to Validity II |
Construct validity, ecological and external validity, the inter-relationships among different types of validity, mundane and experimental realism |
Berkowitz, L., & Donnerstein, E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. American Psychologist, 37, 245-257. Anderson, C. A., & Bushman, B. J. (1997). External validity of "trivial" experiments: The case of laboratory aggression. Review of General Psychology, 1, 19-41. James, L. R. (1973). Criterion models and construct validity for criteria. Psychological Bulletin, 80, 75-83. Landy,
F. (1986). Stamp collecting versus science: Validation as hypothesis testing.
American Psychologist, 41, 1183-1192. |
|
10/20 |
Measurement of Predictor and Dependent Variables |
Content validity, convergent and discriminant validity, problems with dichotomization of constructs, internal structure analysis |
Blanton, H., & Jaccard, J.
(2006). Arbitrary metrics in psychology, American Psychologist, 61, 27-41.
Cohen, J. (1983). The
cost of dichotomization. Applied
Psychological Measurement, 7, 249-254. Maxwell, S. E., & Delaney, H. D. (1993). Bivariate median splits and spurious statistical significance. Psychological Bulletin, 113, 181-190. Campbell, D. T.,
& Fiske, D. W. (1959). Convergent and discriminant
validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Tourangeau,
R. Singer, E., & Presser, S. (2003).
Context effects in attitude surveys: Effects on remote items and
impact on predictive validity.
Sociological Methods & Research, 31, 486-513. Ford, J. K., MacCallum, R. C., & Tait,
M. (1986). The application of exploratory factor analysis in applied
psychology: A critical review and analysis. Personnel Psychology, 39, 291-314. |
|
10/27 |
Mid-Term Exam |
|
|
|
11/3 |
Quasi-Experiments and Non-Experiments |
Nonequivalent groups; time series designs, survey research |
Schaie, K. W. (1965). A general model for the study of developmental problems. Psychological Bulletin, 65, 92-107. Dillman, D. (1991). The design and administration of mail surveys. Annual Review of Sociology, 17, 225-249. O'Neil, K. M., &
Penrod, S. D. (2001). Methodological
variables in web-based research that may affect results: Sample type,
monetary incentives, and personal information. Behavior
Research Methods, Instruments & Computers, 33, 226-233. |
|
11/10 |
Randomized experiments |
Experimental design, random assignment, attrition, stimulus sampling |
Blampied,
N. M. (1999). A legacy neglected: Restating the case for single-case research
in cognitive-behaviour therapy. Behaviour Change,
16, 89-104. Sherman, L. W., & Berk, R. A. (1984). The specific deterrent effects of arrest for domestic assault. American Sociological Review, 49, 261-272. Shadish, W. R., Hu, X., Glaser, R. R., Kownacki, R., & Wong, S. (1998). A method for exploring the effects of attrition in randomized experiments with dichotomous outcomes. Psychological Methods, 3, 3-22. Wells, G. L., & Windschitl, P. D. (1999). Stimulus sampling and social psychological experimentation. Personality and Social Psychology Bulletin, 25, 1115-1125. |
|
11/17 |
Meta-analysis Steve Penrod (Guest Instructor) |
Effect sizes, power, sample size, problems with statistical significance testing |
Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology, 52, 59-82. Friedman, H. (1982). Simplified determinations of statistical power, magnitude of effect and research sample sizes. Educational and Psychological Measurement, 42, 521-526. Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation. Psychological Bulletin, 70, 245-251. Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399. |
|
11/24 |
Moderation and Mediation |
Testing for moderation and mediation; conceptual issues and analytic strategies |
Baron, R., & Kenny, D. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182. Chmura
Kraemer, H. Kiernan, M., James, L. R.,
& Brett, J. M. (1984). Mediators, moderators, and tests for mediation. Journal of Applied Psychology, 69,
307-321. McClelland, G.
H., & Judd, C. M. (1993). Statistical difficulties of detecting
interactions and moderator effects. Psychological
Bulletin, 114, 376-390. Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Establishing a causal chain: Why experiments are often more effective than meditational analyses in examining psychological processes. Journal of Personality and Social Psychology, 89, 845-851.
|
|
12/1 |
Manuscript preparation and reviewing |
Peer-review process, components of a manuscript, reporting statistics, what to include in a manuscript review, how to increase writing productivity |
Silva, P. (2007). How
to write a lot. Platt, J. R. (1964).
Strong inference. Science, 146,
347-353. http://256.com/gray/docs/strong_inference.html Wilkinson, L. (1999). Statistical methods in psychology journals. American Psychologist, 54, 594-604. Sternberg, R. J. (2003). There is no place for hostile reviews. International Journal of Clinical and Health Psychology, 3, 159-161. Ramos-Alvarez, M. M.,
Valdes-Conroy, B., & Catena, A.
(2006). Criteria of the peer-review process for publication of
experimental and quasi-experimental research in psychology. International
Journal of Clinical and Health Psychology, 6, 773-787. |
|
12/8 |
Presentations |
Discussion of favorite research articles |
Article review due in class for those presenting on 12/15 |
|
12/15 |
Presentations |
Discussion of favorite research articles |
Article review due in class for those presenting on 12/8 |
|
12/22 |
Final Exam |
|
|