What Social Science Knows and Does Not Know

Jim Manzi has just published an incisive article in the City Journal titled: What Social Science Does-and Doesn't-Know. In it, he makes a compelling case for experimentation and for the importance of control groups.

He briefly reviews the history of experimentation beginning with Galileo. Then, he provides examples from economics and criminology. He introduces the problem of causal density and writes that it, in effect, dooms many otherwise worthy randomized field trials.

He then moves into the business world citing the success of the credit card company Capitol One. That company answers its own questions via large scale, but relatively economical experimentation. He uses the example of deciding whether to mail customers solicitations in white or blue envelopes. Capitol One simply mailed 50,000 of each color to randomly selected customers and waited to see the results. Since then, other business have adopted similar methods.

He offers three conclusions:

• It is exceedingly difficult to demonstrate that any kind of social program works via traditional randomized and replicated trials. Social evolution with its attendant trial and error process may "trump" those methods.
• Far more social programs fail than succeed, especially programs designed to change the way people think or behave. Incentives work better.
• Most successful programs only lead to modest improvements, but that's ok.

Archive Fever

Marianna Torgovnick recently wrote about archives in the Chronicle of Higher Education (Volume 55, Issue 2, Page B1). We cover archival research in chapter 10.

In her article, she reviews books, exhibits, and video materials and their use as archives. Her view of archives goes beyond our coverage of archives as sources of data for research.

She notes that Foucault's, The Archeology of Knowledge and the Discourse on Language and Derrida's, Archive Fever both note how dependent all of us have become on culturally-based archives such as driver's licenses and visas.

Torgovnick notes how much more archivally creative we have become in our electronic era. The ease by which all of us can now create video materials and post them on the Internet via YouTube and similar outlets has led to an explosion of new archival material.

In addition to these new archival outlets, she points out another phenomenon--pseudodocumentaries, a genre as old as Swift's Modest Proposal and as new as Borat.

So, archival research is much more than simply locating an archive and mining it for its data. On top of that, archives are disappearing too. Torgovnick shows that the usual suspects for the loss of archives, "fire, flood, war, and the passage of time" are also accompanied by other sources of loss such as the unrealized greater fragility of microfilm and microfiche compared to paper. The losses are compounded by the necessity for researchers to sift out "real" archives from those, somehow, less real.

Researchers desire archives that harbor no fiction, and such archives exist. However, lack of fiction is not the same as truth. Torgonovnick shows that people are a necessary link to archives and the story they tell. She worries what false truths our archives might reveal were we no longer around to interpret them.

Like other types of research, archival research requires a careful and diligent research to illuminate the data uncovered and to show what has not been uncovered or what still remains to be found.

Crows Recognize People? Dogs Recognize Cars?

I have a fondness for field research. John Marzluff of the University of Washington and his colleagues observed that the crows they had trapped previously in order to band them for identification seemed to be harder to catch later.

So, they decided to test that hypothesis by conducting a field experiment. Three researchers, Marzluff and two students, then wore a caveman mask and trapped and released seven crows. Later, whenever one of the three wore the map on campus, the crows scolded them severely.

However, whenever they walked on campus wearing another mask (a Dick Cheney mask), the crows ignored the wearers. This, of course, was the control condition.

It seems, thus, that crows recognize people who catch and handle them and remember them later. Other researchers, too, have seen similar behaviors in crows and related birds. See a New York Times article for more information.

Informally, I have conducted similar exercises. I like to watch my dogs when I pull up in a rental car. Invariably, they bark at me viciously from behind the living room windows. They stop barking after I exit the rental car and start to walk toward the front door. Also, I have often sat outside in the afternoon while the dogs walk around in the front yard. I watch carefully for cars while I wait. I don't want to explain to my children why their dogs were run over. Interestingly, when I see my wife a block away and go to protect the dogs, they hear her van approaching and start to run in its direction (something they do not do for other vehicles).

So, it seems that dogs and birds can recognize and remember things about us.

Participant Observation vs. Undercover Investigations

Whenever I teach about participant observation (chapter 10), I nearly always get a question about how participant observation research compares with police undercover investigations.

The first thing I say is that the goal of research is to collect empirical data for scientific purposes. Whereas the goal of an undercover investigation is to collect evidence of a crime. Thus, at the outset there is a big difference in the goals of each.

In participant observation, an IRB must first review the prospectus of the research and decide whether any ethical problems exist. If ethical problems exist, the IRB must provide some guidelines on how to handle them. I like to tell the story of Jim Quinn, a member of the Department of Rehabilitation, Social Work, and Addictions at North Texas State. For his PhD research, he joined the Bandidos motorcycle gang in Miami. That research was also reported in his article:

Quinn, J. F. (1987). Sex roles and hedonism among members of`outlaw' motorcycle clubs" Deviant Behavior, 8, 47-63.

He had to cut his research short because he became convinced that if he remained in the club he would become party to a crime. Ethically, a scientist cannot use the conduct of research as an excuse to participate or be party to criminal acts.

Let's now look at undercover investigations. Recall, the goal here is to find evidence of criminal activity and to, later, prosecute it. Ethics is a concern here too. What are the permissible activities that an undercover officer may participate in? Can the officer smoke a joint? Obviously, officers must lie in order to establish a relationship with a criminal. In sex crime cases, like prostitution, how far can an officer allow sexual contact? Can an officer threaten someone into committing a crime? Can an officer continually offer criminal temptation to a potential criminal? The ethical line seems fuzzier and broader to me here than in participant observation.

All of the questions above are not really part of a typical psychology research course. However, many of the activities that participant observers and undercover investigators take part in are similar. Just remember that their goals are quite different.

SAD?

As we approach the Winter Solstice here in the northern hemisphere, it's a good time to talk about Seasonal Affective Disorder or SAD.

SAD is a real condition caused by increased levels of melatonin. That increase is apparently caused by the longer periods of dark in the winter. Decreased levels of serototin also contribute to SAD. Winter's longer dark periods cause a drop in serotonin production. Light, delivered in high intensities and at the right time of the day, can reverse the rise in melatonin production and the decrease in serotonin, thus, combating SAD. An article in today's New York Times discusses how light, and other therapies, can alleviate SAD.

Another way to fight SAD is to eat complex carbohydrates. Those, in turn, are metabolized into serotonin. Unfortunately, they also add body weight and overeating them is not good therapy for SAD. Sorry.

So, if you are feeling a little blue and hungrier right about now and don't know why, you may have Seasonal Affective Disorder.

Here is a link to some self-assessments for diagnosing SAD, circadian rhythm, and depression. Click on their link "Self-assessments" on the top of that page.

Epidemiology's Methods

Several recent articles have highlighted how epidemiologists conduct research and some have criticized their results.

An LA Times article discusses some of these results, methods, and criticisms. Some of the past results have been 180 degrees apart. Look at the results they cite about coffee and its effects. In 1981, a study concluded that two or three cups of coffee a day tripled the risk of pancreatic cancer. In 2001, another, larger study concluded that the earlier results were not true. Similarly, a 1981 study found that drinking coffee reduced the risk of colorectal cancer. Predictably, a later study in 2005 contradicted it.

Most epidemiological studies are observational. As we discuss in chapter 10, observational studies (e.g., naturalistic observation and participant observation) do not alter the situation under observation. Other ways to conduct epidemiological research is through cohort studies, case control studies (or retrospective), and cross-sectional studies.

The big advantage of epidemiological studies is that they are relatively cheap to conduct. However, they do not allow researchers to make cause-and-effect conclusions about the variables under investigation.

Much better results are provided by randomized clinical trials (or what we call true experiments). Randomized clinical trials are the medical version of true experiments. In a clinical trial, two (or more) groups are created by a random procedure.

Another LA Times article highlights some of the recent research discrepancies discovered by observational methods vs. clinical trials.

A long New York Times article also covers the issues inherent in observational research. That article concludes:

• "All of this suggests that the best advice is to keep in mind the law of unintended consequences. The reason clinicians test drugs with randomized trials is to establish whether the hoped-for benefits are real and, if so, whether there are unforeseen side effects that may outweigh the benefits. If the implication of an epidemiologist's study is that some drug or diet will bring us improved prosperity and health, then wonder about the unforeseen consequences. In these cases, it's never a bad idea to remain skeptical until someone spends the time and money to do randomized trial and, contrary to much of the history of the endeavor to date, fails to refute it."

Our final word echoes our advice in chapter 1, learning about methods is a good thing.

You be good, see you tomorrow...

It's unique for a parrot to have last words: "You be good, see you tomorrow. I love you."

Those were Alex's last words before he went into his cage on Thursday, September 6, 2001. Alex, of course, was Irene Pepperberg's famous African Grey parrot. She had taught him over 150 words, revolutionizing the area of animal cognition in the process.

Alex had been with Pepperberg since 1977 and was 31 years old. Her research continues with two other parrots: Griffin and Arthur.

Rest well Alex, and well done.

Here is a link to a New York Times article about Alex's life. Here is a link to The Alex Foundation, a Web page devoted to Alex and his work.

J. Michael Bailey: The Rest of the Story

On page 334 (in chapter 10), we mention the travails of J. Michael Bailey after he published his book, The Man Who Would be Queen (2003).

Specifically, we wrote:

• Another ethical concern involves informed consent and communication with participants. Because most small–N research involves close and extended contact between researchers and participants, a personal relationship is more likely to develop than in other types of psychological research. The perils of relationship and role confusion in small-N research were evident after Bailey (2003) published a book about transsexuals. After its publication, several of the transsexuals studied accused Bailey of failing to inform them that their interactions with him constituted research. One “participant” claimed that she and Bailey had sex during the time he later claimed he had been observing her as part of his research.

A recent New York Times article follows up on this story and provides more details on what has happened since we wrote the paragraph above.

Bailey has been confronted by several critics, the most prominent is Lynn Conway from the University of Michigan. She maintains a Web page investigating the publication of Bailey's book.

An ethics scholar, Alice Dreger, has conducted an investigation of the case. Her report can be found at http://www.bioethics.northwestern.edu/. It will be published soon in the Archives of Sexual Behavior.

The participant who accused Bailey of sexual misconduct did so five years after the alleged incident. Whether or not the incident actually took place has never been determined.

Two of the other participants who complained about being mentioned in the book were not, in fact, mentioned. The other two said they knew their stories would be in the book.

In short, much ado about something has marked this case. We still stand behind our advice, communication and documentation are essential in small-N research. Bailey could have avoided much of what happened had he made things more clear to his participants at the beginning.

Pointing to Food and Picking Pink

An article in The Economist (August 23, 2007) discusses some recent research in gender differences and the origins of those differences.

Joshua New , Max M. Krasnow, Danielle Truxaw, and Steven J.C. Gaulin conducted a field experiment to investigate whether men or women are better at finding food in a natural environment. They used a nearby farmer's market to test their hypotheses: women will remember the location of previously visited food resources better than will men, and the higher the nutritional value of the food, the better the location will be remembered.

A total of 96 volunteers (41 women and 45 men) each stopped at six of the 90 food stalls in the farmer's market. Later, they pointed (individually) at the six stalls from a location in the center of the market. Women were more accurate, by an average of 9 degrees. Both men and women were more accurate at pointing at the stalls that contained food with higher nutritional value.

These results were originally published in the Proceedings of the Royal Society (the first scientific association, you will recall from chapter 12, p. 384).

Anya Hurlbert and Yazhu Ling asked British and Chinese young (early 20s) men and women to pick their favorite colors as they flashed on a computer screen. While most studies like theirs reveal a near universal preference for blue by both genders, they were able to find a preference for reddish to pinkish hues by women. Participants who scored as feminine on the Bem Sex Role Inventory, also showed a preference for reddish to pinkish hues, regardless of their sex. They did not discover any cultural differences.

These results were originally published in Current Biology. Here is a link to a summary of their article.

The article in The Economist links both of these findings to evolutionary psychology. Women being the gatherers in primitive human hunter-gatherer societies and women being the ones most likely to select edible fruits.

Usability as a Career

In chapter 1 we write about careers and how the research methods course can be leveraged into getting a job or obtaining a promotion. One job we did not mention is that of usability professional.

Usability is a big deal in the age of the Internet, but like most things, it existed before the World Wide Web. However, the profusion of Web pages, computer applications, and new technological goodies like the Blackberry and the iPhone have made us more aware of usability.

The New York Times article "Technology's untanglers: They make it really work" inspired this entry. In that article are examples of usability experts collecting data from users in order to improve their experiences. One expert, Dr. Janice Redish, noted that the main skills she looks for in a usability professional are "solid observation and interview skills." Hmmm....sounds like chapter 10, no?

Just in case you are interested, the article reports that the average salary for a usability professional was $86,500 (in 2005, the latest year reported by the Usability Professionals' Association. Also, many companies are now creating high-ranking positions such as "Directory of Usability."

Meow, researching the cats among us

Cats and humans have lived together for a long time. An abstract in Science (click here) reveals just how long that relationship has lasted and how successful it has been for both parties. A New York Times piece reviews the Science article.

Carlos A. Driscoll and his nine co-authors propose that all 600 million domestic cats in the world are the genetic descendants of five female cats from the Middle East who domesticated themselves about 10, 000 years ago. The domestic arrangement was beneficial to both cats and people. The fact that cats initiated the arrangement helps to explain, in part, why cats are so different from dogs.

The fact that cats are sedentary, our propensity to take our cats with us when we move, and the ease of identifying cat genes via their coats has revealed much about human population movements. See "Cats and commerce" in the November, 1977 Scientific American or "Pussy cat, pussy cat, where have you been" in the August, 1986 Natural History. Both those articles show how we can use our long relationship with cats to discover facts about ourselves (and them).

Narrative Psychology and Methodology

Another New York Times article nicely illustrates the interplay of qualitative and quantitative research in exploring a complex psychological topic: whether or not personality can be explored via first-person accounts. This is, of course, a throwback to some of the earliest ideas in scientific psychology.

The research summarized in the article and further elaborated upon below re-opens many questions asked by early psychologists. However, the methodological lessons learned since psychology's early days are fully understood by these researchers.

Thus, McAdams et al. (2006) (see full references below) find that narrative accounts of one's life show evidence of continuity. In other words, they check to be sure that the phenomenon they are studying is reliable over time. McAdams and his co-workers report on research in which college students write detailed accounts about their lives three times (initially, at three months, and at three years). They found continuity for "narrative complexity, and...emotional tone."

Similarly, McLean and Pasupathi (2006) find that extraverts were more likely to engage in collaborative narration. They test an independent variable (extraversion/introversion) and find that it predicts an effect under two separate conditions. McLean and Pasupathi report on two studies, the first on self-defining memories and the second on everyday narration. Their hypothesis, that extraverts were more likely to engage in collaborative narration was supported by both studies.

Adler et al. (2006) show that depressive thinking and concern over contamination each independently predicted depression. They, too, examine independent variables (self-thoughts about depression and contamination) and find that each predicted depression. Adler and his collaborators used the CAVE (the Content Analysis of Verbatim Explanations) to evaluate 70 life interviews from midlife adults. They found that depressogenic attributional style and themes of contamination each independently predict depression and low satisfaction with life.

Kross et al. (2005) attempt to look at the difference between rumination and adaptive reflection. Here, the independent variable was created by dividing participants into two groups depending on whether they asked themselves 'Why' questions. Those who did ask themselves 'Why' experienced cooler emotional reactions when thinking about past memories. In two experiments, the authors attempt to "disentangle" rumination from adaptive reflection. Their results suggest that individuals who ask 'Why?' were more able to re-experience emotional experiences in a cooler manner and without reliving the experience.

Finally, Libby et al. (2005) examine first-person and third-person memories of emotional events. They find that change is more likely to accompany third-person retrospective accounts. Again, they used participants' self-selection of first- or third- person accounts to create groups. Libby and her colleagues studied first-person vs. third person memory perspectives in five separate studies (looking at motivation, goals, instructions, and self-esteem). They found that third-person perspectives were more likely to produce judgments of self-change in the context of looking for change. However, third-person judgments were less likely to produce judgments for self-change in contexts looking for continuity.

For further reading:

Adler, J. M., Kissel, E. C., & McAdams, D. P. (2006). Emerging from the CAVE: Attributional style and the narrative study of identity in midlife adults. Cognitive Therapy and Research, 30, 39-51.

Kross, E., Ayduk, O., & Mischel, W. (2005). When asking 'Why' does not hurt: Distinguishing rumination from reflective processing of negative emotions. Psychological Science, 16, 709-715.

Libby, L. K., Eibach, R. P., & Gilovich, T. (2005). Here's looking at me: The effect of memory perspective on assessments of personal change. Journal of Personality and Social Psychology, 88, 50-62.

McAdams, D. P., Bauer, J. J., Sakaeda, A. R., Aniyidoho, N. A., Machado, M. A., Magrino-Failla, K., White, K. W., & Pals, J. L. (2006). Continuity and change in the life story: A longitudinal study of autobiographical memories in emerging adulthood. Journal of Personality, 74, 1371-1400.

McLean, K. C., & Pasupathi, M. (2006). Collaborative narration of the past and extraversion, Journal of Research in Personality, 40, 1219-1231.

Unfairness and heart attacks

A recent article in the Los Angeles Times discusses the relationship between believing one was treated unfairly and subsequent heart attacks and angina (chest pain).

Civil servants in England were asked, "I often have the feeling that I am being treated unfairly." At the time of surveying, none of the approximately 6000 respondents were in poor health or showed signs of heart disease.

Ten years later, for those who reported high levels of unfair treatment, 387 had either died or had been treated for angina or other heart problems.

Respondents who reported less unfair treatment had lower levels of cardiac events.

The authors interpret these results as evidence for a possible relationship between cardiovascular and mental health. The mechanisms for the relationship are indirect. People who believe they have been treated unfairly are more likely to drink, smoke, overeat, and engage in other similar risky behaviors.

Wagging Tails

A recent New York Times article summarizes research on differences in tail wagging by dogs. The article was published in the journal Current Biology recently. This link gives the doi locator for the article. Here is the citation:

Quaranta, A., Siniscalchi, G., & Vallortigara, G. (2007). Assymmetric tail-wagging responses to different emotive stimuli. Current Biology, 17, R199-R201.

Because we each first came to psychology as animal researchers, we naturally perk up when we find articles that stimulate our original interests. This particular article describes observational research conducted on pet dogs which indicates that their tail wagging direction differs depending of whether the stimulus causing the wagging is familiar or unfamiliar.

The 30 dogs observed were more likely to wag their tails to the right when presented with a familiar stimulus (their owner), but when presented with a large, dominant dog (a Belgian Malinois ) they were more likely to wag their tails to the left. The two other stimuli used, an unfamiliar human and a cat also led to right-side tail wagging, but with a lower amplitude.

Also of interest here is their method, naturalistic observation. From initial, unstructured observations, the authors conducted a more formal and controlled experiment. Their research serves as a good example of how scientists move from observation to experimentation.

NY Times article: Tracing the cigarette's path from sexy to deadly

Here's a link to a New York Times article (2007-03-20): Tracing the cigarette's path from sexy to deadly. It details how long it took people and society to realize that cigarettes are dangerous.

National Irag Survey

ABC News, in conjuction with BBC and ARD TV (Germany) has conducted another opinion survey in Iraq using a random sample. Here is how they selected respondents (see http://abcnews.go.com/US/story?id=2954886&page=1 for complete details):

"Four hundred and fifty-eight sampling points were distributed proportionate to population size in each of Iraq's 18 provinces, then in each of the 102 districts within the provinces, then by simple random sampling among Iraq's nearly 11,000 villages or neighborhoods, with urban/rural stratification at each stage."

"Maps or grids were used to select random starting points within each sampling point, with household selection by random interval and within-household selection by the 'next-birthday' method. An average of five interviews were conducted per sampling point. Three of the 458 sampling points were inaccessible for security reasons and were substituted with randomly selected replacements."

One measure of the success of the survey was that none of the interviewers were killed or injured during the course of the research. Comments from the pollsters can be viewed at:

http://abcnews.go.com/US/story?id=2954867&page=1

Some of the main results are summarized below:

• My own life is going well: 2005/71% said yes and in 2007/39% said yes
• I expect my life to get better: 2005/64% said yes and in 2007/35% said yes
• My own neighborhood is not safe: 2005/36% said yes and in 2007/74% said yes

The survey was conducted by D3 Systems of Vienna, VA, USA.