The Ten Elements of a Good Social Science Research Project

Amber Boydstun University of California, Davis Updated September 20, 2009 Broadly speaking, a good social science research project consists of ten main elements, each performed well: Puzzle What is the political phenomenon that is puzzling enough to warrant investigation? The puzzle you identify should be clear and compelling. Explain to your reader, in a nutshell, why this question interested you enough to study it in the first place. Avoid personal anecdotes (e.g., Ive wanted to study the effects of negative campaign ads on voter turnout ever since I was a campaign volunteer in 2000). Instead, think Sherlock Holmes or, better yet, Nancy Drew (e.g., We might think that negative campaign ads would prove beneficial for the candidate running the ad, yet in district X where twice as many negative ads were televised during the 2000 campaign as in any previous campaign, voter turnout plummeted). See how important the yet was in that example? Identify a mystery, do whatever you can to set the spooky-exciting mood music of the mystery, then ask the reader: Whats going on here? Whats up with that? Oh, and you should be able to do all this set-up quickly and concisely. You want your reader to say you had me at plummeted. Research Question What is the primary question suggested by this puzzle that you try to answer? This question should be testable, meaning that in most cases it should be phrased in terms of causal relationships, even though often youll only be able to prove correlation, not causation. The most common research question variants include: What is the effect of X (a key independent variable) on Y (a dependent variable)? What factors (one or more independent variables) lead to Y (a dependent variable), and what is the specific effect of each? What is the relationship between Y1 (one key variable) and Y2 (another key variable)? Does Y1 drive Y2? Does Y2 drive Y1? Or do Y1 and Y2 have a mutually-reinforcing relationship? Significance So what? Why does this research question matter? If you find one answer (e.g., that X has a positive effect on Y), what will that result tell us about the world that we didnt know before? And just as importantly, if you find another answer (e.g., that X has a negative effect on Y, or that X has no effect on Y), what will that result tell us about the world, again that we didnt know before? You should have captured your readers interest with your puzzle, but here you need to convey to them the full weight of your question. Convey how your research question holds implications for the immediate political phenomenon under investigation, and consider also discussing the broader implications your study holds for bigger questions beyond the scope of this particular study.

Literature Review What insight does past research give you into this question? In general (but with notable exceptions!), published research got published because it was good research. This literature provides the building blocks for your study, whether you like it or not. Thus, you need to address all the key points that relevant past studies have to say about your question. Extant findings that correspond with your understanding of the world will be useful in supporting your theory. But you must also address research that runs counter to your understanding of the world. Your study will only be believable if you can situate your work within the body of knowledge that already exists and, in particular, if you can reconcile your theory and findings with any previous contradictory conclusions. Theory What do you think is the answer to your question, and why? Explain your understanding of the world that leads you to this expectation. Be logical, systematic, and detailed. Think about the stages of a mathematical proof, and proceed in similar syllogistic fashion. If you want to say that A implies B, you need to explain why. And before you jump to talking about how C implies D, you need to explain how B leads us to C. Be sure to describe in detail the specific causal mechanisms you think are at work. Even if the phenomena you address in your theory are amorphous and difficult to measure (like individuals cognitive perceptions, social trends, or governmental responsiveness), you should be thinking (and writing!) in terms of concrete X causes Y links, like billiard balls hitting each other in sequence on a pool table. Hypotheses What (is)are the testable, falsifiable, and explicit hypothes(is)es that derive from your theory? A testable hypothesis is just like it sounds: a hypothesis that can be tested. For example, the statement Our universe is surrounded by another, larger universe, with which we can have absolutely no contact is non-testable, as is the statement Negative campaign ads are evil. Many hypotheses are theoretically testable but not practically so. For example, while it would technically be possible to survey every human being on the planet, the hypothesis All people are happy is effectively non-testable. A falsifiable hypothesis is a hypothesis that can be both proven (i.e., proven to be true) and disproven (i.e., proven to be false). For example, while the hypothesis Intelligent life exists on other planets is testable, it is not falsifiable because there is no finding that would prove it false. The hypothesis could be proved true, if a space probe reveals incontrovertible evidence of intelligent life on another planet for example. But the absence of evidence of intelligent life does not disprove the hypothesis. Even if thousands of years go by without evidence of life on other planets, it doesnt necessarily mean none exists. Practically speaking, it can be very difficult to develop hypotheses about politics that are universally falsifiable, but thats our goal nonetheless. For example, the hypothesis Negative campaign ads always decrease voter turnout is falsifiable, since a single study showing that voters exposed to negative campaign ads turned out to vote at normal levels would disprove it. But the hypothesis Negative campaign ads decrease voter turnout is more difficult to test universally. A single study showing that voters exposed to negative campaign ads turned out to vote at normal levels would only prove that negative campaign ads dont decrease voter turnout in the particular conditions of that study. It may be Boydstun: Elements of a Good Research Project Page 2 of 5

the case that under other conditions (e.g., when its not raining, in the case of less educated voters, when the negative ads used religious rhetoric, etc.) negative campaign ads do indeed decrease voter turnout. Thus, much of social science research is focused on parsing out the specific conditions under which X does or does not lead to Y. In this context, it is especially useful to articulate our hypotheses in as falsifiable a form as possible (e.g., Negative campaign ads employing religious rhetoric decrease voter turnout among less educated voters). Conceptual Hypotheses Although most authors only present one set of hypotheses (and some authors choose not to list their hypotheses at all), in fact all scientific research projects contain two kinds of hypotheses, whether explicitly stated or not: conceptual and operational. Conceptual hypotheses are, just as they sound, composed of concepts, and thus by definition impossible to test directly. These hypotheses should probably appear at the end of your theory section, as a way of summarizing the theoretical expectations about the world that derive from your larger theoretical discussion. They should be specific to your research question but phrased in conceptual terms, meaning that you should compose your conceptual hypotheses not around the specific study you are undertaking but for the phenomenon in general. For example, Negative campaign ads decrease voter turnout is a good conceptual hypothesis. It succinctly communicates an expectation about the world, but it is not immediately testable; being conceptual in nature, by definition it cannot be measured directly. In other words, it would be difficult to know precisely what to look for in the findings section of a study in order to know whether this hypothesis is supported. Operational Hypotheses While testability and falsifiability are the most important criteria of a hypothesis, once those criteria are met the practical purposes of social science research require explicit expectations about the litmus test you will be employing in your particular study. Thus, it is important to translate each conceptual hypothesis into an explicit operational hypothesis that succinctly states the predictions that your theory would lead us to expect from the results of your current study. For example, our conceptual hypothesis, Negative campaign ads decrease voter turnout, is both testable and falsifiable, but it isnt very explicit. It doesnt tell us concretely what to look for in our study in order to know whether or not our theory is supported. A good operational hypothesis tells the reader exactly what you expect the evidence in your study to show. For example, the operational hypothesis Residents in district X where negative campaign ads were aired on network television in October of 2004 should be significantly less likely to have voted in the 2004 election than residents in district Y where no negative campaign ads were aired is a testable, falsifiable, and explicit operational hypothesis. Bingo! Data What evidence do you examine in order to test your hypotheses? Data refers to any information that can be systematically observed. Generally, social scientists divide data into two categories: quantitative (information that can be expressed numerically) and qualitative (information that cannot be expressed numerically). Neither quantitative nor qualitative data is inherently better. Both can be used well, and both can be used poorly. In fact, the underlying data sources for qualitative and quantitative data are often the same. What distinguishes quantitative from qualitative data is how the researcher chooses to evaluate it. Boydstun: Elements of a Good Research Project Page 3 of 5

When a researcher assesses a data source systematically by measuring specific variables, the resulting data is quantitative. When a researcher assesses a data source systematically by developing a non-numeric evaluation of the evidence, the resulting data is qualitative. For example, if a researcher wants to compare the use of political rhetoric in the television ads of competing Presidential candidates, the researcher could collect quantitative data by identifying specific criteria on which to assess the different ads (war-mongering, mudslinging, patriotism, etc.) and then measure corresponding variables within each ad (e.g., the number of times the candidate mentions her support of a war, the number of times the candidate says something bad about her opponent, the number of times the ad shows patriotic imagery like an eagle or a flag). Alternately (or additionally), the same researcher could collect qualitative data from the same ads through a narrative evaluation of their similarities and differences, in a manner similar to a sophisticated compare and contrast essay. In this case, the researcher would still develop specific criteria on which to assess the different ads, but the criteria would be subjectivethat is, non-numericand the researcher would evaluate the criteria based on personal observation rather than objective measurement. Examples of quantitative data include: public opinion poll results, number of troop deaths, number of marches or riots that occur in a given social uprising, number of newspaper stories that portray a candidate in a positive light as opposed to a negative light, and so on. Examples of qualitative data include: a descriptive summary of public attitudes, a narrative account of troop loss, a historical account of marches or riots, a descriptive analysis of the use of positive and negative portrayals in newspaper coverage of a candidate, and so on. Research Design Exactly what steps will you take to test your hypotheses? The research design is the glue of your project: the roadmap that governs how you will collect, analyze, and evaluate your data. Without a good theory, theres not much point in developing a research design. But assuming you have a good theory, the worth of your project will rest on your research design. Your research design can be basic, complex, traditional, innovativewhatever is most appropriate for your project (though all else being equal, parsimony is a real virtue in this area). But whatever form your research design takes, it must be scientific. In the development stages, examine and then re-examine your research design for flaws. In the implementation stages, record everything. Be dogmatically systematic. And make sure every single thing you do, from data collection to final analysis, can be easily replicated by a future researcher based on nothing but your research notes. As with data sources themselves, research designs can be broadly categorized into qualitative and quantitative forms. In fact, as the discussion about data above should illuminate, in many cases the research design is what determines whether a given source of data will be used qualitatively or quantitatively. Qualitative research designs include discourse analysis, historical interpretation, and case studies. Quantitative research designs include experiments, quasiexperiments, surveys, large-N analysis (e.g., using cross-sectional or time series data), content analysis, and theoretical models (e.g., formal and computational). Each of these research methods is associated with best-practice guidelines for how to employ the design appropriately. And so once youve selected your research design you should follow these guidelines carefully Boydstun: Elements of a Good Research Project Page 4 of 5

to uphold the scientific method. But which research design you select should be based on the particular goals and parameters of your project. Think critically and creatively about what steps would best allow you to test your hypotheses, and then choose the research design that best meets your needs, tailoring it as necessary. Findings & Conclusion What results do you find, and what can you conclude from these results? If youve set up your hypotheses clearly and developed an effective and efficient research design, it should be simple to tell whether you find support for your hypotheses or support for the null hypothesis. Whether youre using a qualitative or quantitative approach, you should take steps to insulate your findings from any bias that might creep into your interpretation as you cross your fingers that your hypothesis is true (being proven wrong happens to the best of us). Implications Beyond the specific importance of your findings in terms of confirming or disproving your hypotheses, what do your findings suggest about the world more generally that we didnt know before? Your implications should link back to your earlier discussion of the significance of your research question, with the important difference that now, because of your study, were a small step closer to understanding this political phenomenon than we were before. Explain exactly what that step has gained us, what it hasnt gained us, and why it all matters. For example, if you confirm your hypothesis that Residents in district X where negative campaign ads were aired on network television will be significantly less likely to vote than residents in district Y where no negative campaign ads were aired, the most immediate implications of your study are what this finding suggests about how negative campaigns affect voters more broadly. And you can even discuss broader implications, such as what this finding suggests for how people process political information in general, or what the ideal strategy would be for a political candidate wanting to depress voter turnout among citizens of the opposing party, as long as you remember that the further away you get from your specific findings, the thinner the ice becomes. So use this section to remind your reader why your study is exciting and important and to connect the dots to show how your findings actually speak to much bigger questions than the specific one you addressed in your study. But remember that the only thing youve proved is that your specific hypothesis is (or is not) true.

Boydstun: Elements of a Good Research Project

Page 5 of 5