Breaking Down Research Designs

After a less-than-stellar set of test grades on chapters 7 and 8, I decided to re-explain the characteristics of research design. The test question was:

What is a design? What are its components?

In chapter 2, we state (p. 44): "The design of a research project includes the number of groups, how they are treated, and how the behavior is measured. The dependent variable, the independent variable, levels of the independent variable, and how extraneous variables are controlled are all aspects of research design."

In chapter 7, we introduce between-subjects designs and the extraneous variables of selection, differential attrition, and diffusion of treatment. In chapter 8, we introduce within-subjects designs and the extraneous variables of testing, instrument change, history, maturation, and regression (to the mean).

Let's break down design even further.

RESEARCH Type

• Experiment
• Quasi-experiment
• Field Study
• Naturalistic Observation
• Participant Observation
• Case Study
• Interview
• Focus Group
• Oral History
• Archival Study
• Small N Study
• Other
  

NUMBER of Groups (N)

• One
• Two
• Three
• Four
• More

GROUPS

• Between Subjects
• Within Subjects
• Mixed

VARIABLES

• Independent (number, levels)
• Dependent (number, quantitative or qualitative, continuous or discrete)
• Extraneous (how controlled or not)

EXPLORATORY DATA ANALYSIS & STATISTICS

• Graphs
• Descriptive Statistics
• Correlations
  
• Confidence Intervals
• NHST Tests and Significance (e.g., t-test, ANOVA, chi-square)
  
• Nonparametric Tests (e.g., Mann-Whitney, Wilcoxon Matched-Pairs Signed Ranks, Spearman correlation coefficient)
  
• Effect Sizes

CONTROLLING EXTRANEOUS VARIABLES

• Via design
• Via procedures

ETHICS

As we note in chapter 3, any design that fails to properly follow the current ethical standards is, by definition, a bad design.

So, there are many aspects to research design. Careful researchers devote much time toward perfecting their design and then pilot testing it before collecting data for real.

Are there laws of psychology?

One of the factors that led Mjøset (2001) to differentiate social science from physical science was that the latter had succeeded in formulating laws of nature such as the second law of thermodynamics. (Those laws are undisputed, universal statements about how nature works.) He noted that many early psychologists hoped to formulate similar laws of nature within psychology. Unfortunately, no such laws have yet been discovered, nor may they ever be. In some ways, then, the physical sciences have far surpassed the social sciences because of the presence and reality of physical laws. The situation is somewhat similar between the biological sciences and the physical sciences too. It is impossible to find biological laws either. Some social scientists, notably Merton (1949), simply decided to continue practicing science and forgo any hope of discovering universal laws. In psychology, a similar story exists. It is impossible to find results that apply in any situation. Instead, results must be carefully couched within a disciplinary, subdisciplinary, or finer-grained contexts. There are no laws of psychology.

All is not lost however. Some results have stood the tests of time and of multiple replications. While the examples to follow fail to reach the criterion of a scientific law they serve to illustrate real and reliable psychological data. The first example is Ebbinghaus’ research on human memory. His 1885 book, Über das Gedächnis (Concerning Memory) caused a sensation when first published. He was the first to show the relationship between memory and the passage of time. Simply put, we forget much more quickly soon after learning and forget much more slowly thereafter. The figure below shows the relationship between memory and time.

Ebbinghaus’ discovery does not rise to the level of a scientific law because other conditions (e.g., practice) can alter the relationship between memory and time. So, the relationship is real and reproducible but it does not apply to all types of memory. A second example is Shepard and Metzler’s (1971) mental rotation research. In a laboratory setting, they projected pairs of geometric stimuli to human participants. While the stimuli were projected in two dimensions, they were designed to convey information in all three dimensions. Participants had to decide quickly whether the two stimuli were alike or different. The stimuli which were alike were presented from 0° up to 180° of rotation from each other in any plane. Like Ebbinghaus, they discovered a remarkably straightforward relationship between the amount of rotation and the time it took to decide. As the rotation approached 180°, participants took longer to decide. Moreover, the relationship was linear. See the figures below for examples of the stimuli used and the results Shepard and Metzler found. Pigeons, too, have been tested for their abilities to



mentally rotate objects. Unlike humans, pigeons are able make accurate mental rotations from various points of view (Köhler, Hoffman, Dehnhardt, & Mauck, 2005). Humans perform mental rotations best while in a normal, upright position. Pigeons, on the other, hand perform mental rotations equally well regardless of their spatial relationship to the stimulus. Flying, apparently, affects how pigeons make mental rotations. Thus, the results show that different species make mental rotations differently. Again, while the results of mental rotation experiments are replicable, they are not universal. The species tested makes a difference.

References

Köhler, C., Hoffmann, K. P., Dehnhardt, G., & Mauck, B. (2005). Mental rotation and rotational invariance in the rhesus monkey. Brain, Behavior, and Evolution, 66(3), 158-166.

Merton, R. K. (1949). Social theory and social structure. Glencoe, IL: Free Press.

Mjøset, L. (2001). Theories: Conceptions in the social sciences. In N. J. Smelser & P. B. Baltes, (Eds.). International encyclopedia of the social and behavioral sciences. 23, 15,641–15,647.

Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171(3972), 701-703.

Psychology's Borders

Gray (2008) notes that psychology fits neatly in the middle of nearly every academic discipline. The figure below shows how Gray places psychology in a central position with the natural sciences, social sciences, and humanities arrayed around it. He adds, “It would be impossible for people from any other department to draw a diagram nearly as elegant as mine that put their discipline in the center.” (p.30) I agree. The centrality of psychology creates borders between it and many nearby disciplines. Four disciplines: sociology, biology, computer science, and philosophy have especially intimate borders with psychology. Over time those borders have moved as well. Those border realignments are historically important to understanding 21st century psychology.

Psychology's central position in the academy is probably one reason why psychology courses and the psychology major are so popular.

Gray, P. (2008). The value of Psychology 101 in liberal arts education: A psychocentric theory of the university. Observer, 21(9), 29-32.

Zeitgeist of the Middle Ages

I spent the Christmas holiday working on a book proposal and playing golf (also a little tennis). One of the things I'm sending publishers is a description of what it might have been like to live in the Middle Ages. The textbook is for the history of psychology course, in case you are wondering.

For those who do not know, zeitgeist is a German word that describes what it feels like to live in a particular time and place.

Here goes:

The Zeitgeist of the Middle Ages

Religion was probably the most important difference between then and now. For during the medieval period, religion thoroughly permeated every aspect of life to a degree almost unimaginable today. Yet, at the same time, daily life contradicted those same religious principles. Violence was endemic and justice uncertain. Life was seen as a temporary state, a trial leading to eternal salvation or damnation. Thus, efforts were few to reform social structures or to change behavior because, for the blessed, salvation awaited; and, for the wicked, damnation. Religion also stifled creative thought because God's plan had already been revealed. All human explanations had to account for Biblical truth and for religious dogma. The world and humankind were unique reflections of God's creation. The gradual sense of a loss of uniqueness caused later by Copernicus, Galileo, and Darwin were still far in the future.

Throughout the period, evidence of earlier Roman civilization still stood. Early in the period, a belief that the world was in decline must have been universal. Later, however, as new towns grew, and great cathedrals and castles were built such feelings probably abated some. However, our notions of constant and universal progress would probably have been unrecognizable. Instead of progress, stasis was a hallmark of the period. But, some technological change did take place, slowly. For example, the invention of the chimney allowed for the heating of individual rooms and served to separate the classes from the great common rooms before, where all huddled overnight around the only fire. Town clocks were built, and changed forever perception of time. In the service of war, metallurgy advanced.

Socially, the world was highly structured. The three estates consisted of the clergy, the nobility, and the rest of humanity. Within each estate, of course, large differences existed. The clergy had a special role given the religiosity of the period. The nobility are our main source of information about the period, because of their status. Comparatively little is known about the daily existence of peasants, but their lives can probably be safely assumed to reflect best the stasis of the period. Later in the period, as towns and commerce grew, a middle class developed. Jews, excluded from "proper" occupations, suffered throughout the period. Massacres, exiles, and discrimination were both common and viewed as righteous, given the Jews' alleged role as "Christ killers" and their subsequent refusal to adopt Christianity. In the same light, the Crusades seemed to make abstract sense, even though in a practical sense they were no testimony to Christian principles.

The universities of the Middle Ages were established to reconcile philosophy and theology. Early on in the history of universities, many of the religious orders opened houses of study. The course of study at the medieval university was much different than today's curriculum. The trivium, or introductory curriculum, consisted of three courses: grammar, logic, and rhetoric, while the quadrivium, or advanced curriculum, consisted of geometry, astronomy, arithmetic, and music. Books were all produced by hand and were, consequently, rare. Relatively few attended the university, and those came from the clergy or the nobility. (As an aside, the oldest universities are: Bologna, Paris, and Oxford)

So, life was far different during the medieval period than it is today. Yet, some of its vestiges still remain, such as superstitions and nursery rhymes. Zeitgeist yourself back to a medieval village. Imagine the smell of raw sewage flowing through the gutter in the center of the street. Think of the rigid class structure, the status of women, and the lives of children. Finally, examine how modern culture is descended from medieval culture, and what things have changed and what have not.

Fall 2008 Completed Research Projects

Here's another batch of student research projects. This particular class did a very good job planning and executing their research over a two-semester time span.

• Does noise affect the concentration and response time of college students? (A classic lab experiment in which participants had to complete a reading test while either listening to the sound of a jackhammer at 75db. No significant difference found.)
• Baptist feelings toward science at a small Southern college over a 47-year span (A replication of a survey conducted in the early 1960s. Interestingly, 2008 respondents were significantly less likely to believe that: "It is possible to harmonize modern scientific findings with religious concepts?")
• Reasons Facebook users accept friend requests from strangers (A low response rate led to insignificant results, nonetheless this remains an important topic.)
• What makes a woman stay with her abuser? (Women from a local shelter and college students who responded to an e-mail request were surveyed. A small N led to nonsignificant results.)
• Female perceptions of male intelligence based on first names (A partial replication of a 1993 study discovered that men with younger generation names [e.g., Matthew, William, and Ethan] were perceived as more intelligent than men with older generation names [e.g., Harry, Don, and Fred].)
• Are cigarettes purchased more by males or females: Age and race effects (A field experiment at a local convenience store found that older, White males purchased the most cigarettes, younger, White males the most Skoal, and younger, Black males the most cigars.)
• Who are more depressed: Black or White men? (A survey study using the Beck Depression Inventory found no differences with two groups of 25 male colleges students.)
• A small campus study on classroom seating due to student gender (An observational study of where students sit in class by instructor gender, student gender, instructor race, and student race found that females are more likely to sit near the front of the class and more so when the instructor, too, is female.)
• Are children in two-parent households more academically successful? (Conducted at a local school system, the research found no difference in student GPAs by household type.)
  

Please contact me if you are interested in further information about any of these research projects.

An Academic Pack Rat

I was reading Mischel's column in the November APS Observer. In that column, Mischel speculated about why psychologists reviewing grants are so tough on each other. He noted that it is relatively easy to judge the methodology of a study, but that it is a lot more difficult to judge the importance of the work within the larger context.

Mischel referenced a 1973 American Psychologist article by Cartwright, Determinants of scientific progress. So, I walked out of my office into the lounge where we keep our American Psychologists and picked up Volume 23, Number 3 (March, 1973) and opened it up to page 222. Our in-house collection goes back to 1955 and is largely complete. It represents the personal collections of several faculty over many years.

Next, I read Cartwright's article which is about the risky shift and how it became an important topic in social psychology. Cartwright wrote (p. 223), "Interest in the field [the risky shift] was heightened further by the publication of a popular social psychology text by Brown (1965), which devoted an entire chapter to this research and proposed an ingenious explanatory scheme to account for the major results known at the time."

It just so happens that when I took social psychology in 1970 as my second-ever psychology course, Brown's text was the one used. I walked over to my bookshelf, picked up the volume and found the chapter, Group Dynamics, and read the several pages on Stoner's original research on what is now called the risky shift.

Then, I went to my one of my file cabinets and retrieved my notebook from that 1970 class. On May 4, 1970 we discussed in class what Brown called Stoner problems in the text. Certainly, I did not recall that class or our discussion. A few pages later, I noted Stoner's name among the others the class was supposed to know for the final exam. (The other names for that chapter were Sherif, Asch, and Bales.)

While scanning my old notebook I was struck by how many topics that had been covered in that class were now totally familiar to me: Calhoun's rat crowding study, LeBoeuf's elephant seals, Harlow's attachment research, Heider's balance theory, Gestalt psychology, the founding of the Royal Society, and many more.

When I first read Mischel's column and Cartwright's article, I wondered what I was doing in March, 1973. I recall I was a senior finishing up my undergraduate degree in psychology at the University of Baltimore. Certainly, I was not thinking about what makes a particular piece of scientific research important. The other thing I thought of was how nice it is to have old materials at hand. It reinforces my pack rat tendencies.

New-ro Psychology

An article by David Glenn in the Chronicle of Higher Education, Psychology departments are changing their behavior (December 5, Volume 55, Issue 15, Page A1) discusses how research in psychology is changing because of neuroscience.

As we have noted earlier, the practice of science has become more of a team sport. Neuroscience is suited to groups of scientists working together because of its inherent complexity, large equipment costs, and necessity for specialization.

Like physics and biology before, psychology is now becoming "big science" and directors of research projects manage large budgets and supervises teams of scientists and assistants. Fortunately, according to Glenn, the emergence of neuroscience has, mostly, led to cooperation with older, traditional forms of behavioral psychology.

One exception, however, has been grant funding. Since 2004, the National Institute for Mental Health has changed its research priorities and now tends to fund research that has neuropsychological or genetic components. (Here is an article from APA on that topic.)

Glenn quotes Alan Kraut, APS's executive director, "Everybody, I think, would recognize that behavior is ultimately the result of biological, environmental, and genetic processes...But that doesn't mean that every study needs to have a biological component."

Clearly, psychology has entered a new era, one characterized by the search for the neurological causes of behavior. This is not a bad thing. However, it means that those who wish to research psychology will have to adapt and learn new ways to work together.