Citation
Vogt, W. Paul et al. When to Use What Research Design. New York: The Guilford Press, 2012.
Personal Connections and Reflections:
As usual, I found the organization of the chapters to be clear and concise. Observational research is practiced in museum studies as a way to gauge visitor engagement. I have not utilized the research method directly, but as a curator, I often relied on the assessment of volunteer docents who relayed observations about visitors in our gallery space. These reactions were an important way to assess the level of interest in exhibitions and measure the effectiveness/impact of visual content. I look forward to considering observational research as a potential method for future dissertation research.
Chapter 4 – When to use Naturalistic and Participant Observational Designs
Key Points and Terms:
Observational research means studying social phenomena as they occur. Researchers gather evidence rather than generate it.
Chapter 4 considers when to use observational designs, distinguishing between naturalistic and participant observational design options. Observation is the root of virtually all research and should be considered when a researcher wants to study aspects of lived experiences from interaction with others.
The two main types of observational research are naturalistic and participant. In naturalistic observation, researchers seek to keep their influence on the observed phenomena to a minimum; they want to be as unobtrusive as possible. However, there are not actually very many opportunities to observe people without having any interaction with them at all, and ethical concerns for privacy often require that you make your presence known. Participant observation is much more common in social science research and the level of participation will vary.
· Naturalistic/covert research “pure” type of naturalistic observation.
· In naturalistic/overt research, your role as researcher is explicit, but you do your best to remain in the background while you observe.
o Naturalistic observation is an effective design choice when: You can take advantage of events unfolding in the public sphere. You are making initial entries into the field to explore sensitizing concepts or variables.
· In participant observation, the researcher is a witness and engaged other in the action and discourse. Can also be covert or overt.
o Participant observation is an effective design choice when: You want to see what a phenomenon looks like from the inside. You are particularly interested in diverse perspectives about what you are studying. You want to study something over time as it unfolds, perceptions emerge, and meanings are ascribed. You want to influence what you are studying.
It makes sense to turn to observational designs when your research questions ask about variation, context, detail, and depth.
· Variation means those phenomena or concepts that represent something new, or that differ from each other in observable ways.
· Context is settings/practices that consists of various circumstances and facts that both inform observations and affect the situation you are observing. An interrelated set of varying conditions that include setting, time (including historical time), practices, and other attributes in the observational milieu.
· Detail is the “thickness” of description.
o “thick” description of social phenomena is foundational in observational research. Thick description means portraying a phenomenon, such as a behavior, in context.
· Depth means exploring something new (versus studying established variables).
Observation is an effective research design choice when your research question leads you to: 1. Study social, cultural, psychological, or political processes as they unfold 2. Identify, develop, or refine sensitizing concepts or variables 3. Cultivate a rich or thick description within a particular context 4. Uncover or explore causal mechanisms or recognize interactive links between and among variables.
· Without observation, causal mechanisms are difficult to establish regardless of the research methods used, including experiments and statistical analyses of all kinds.
· Observation often identifies the mediating variables that explain the regularities. Also, observational research may lead to identifying variables and their potential relationships that subsequent experimentation can test.
· Observational designs are more improvisational and require gathering data with all its real-life complexity, as compared to data that are generated according to protocols developed in advance of data gathering.
Chapter 10 - Searching and Sampling for Observations
Key Points and Quotes:
This chapter provides an overview of how research questions shape both in-advance and in-the-field decisions about sampling.
Authors suggest three different lenses that you can use to help in sampling, starting with three basic considerations for sampling observations, determining: (1) how appropriate or relevant a sample is; (2) how to access authentic opportunities to witness phenomena represented in our research questions; and (3) how to balance various practical and resource concerns.
· appropriateness in observational research uses sampling logic determined by two factors: the relevance and representativeness of observation sites and particular occasions for your research questions
o Three factors potentially bias the sample in ways that call authenticity into question. The first is a lack of breadth. The second is insufficient attention to changes over time and how these may bias the sample. Finally, a lack of depth or variations in the depth of data collection across the sample may be a source of distortion
o Your job as researcher is to find the examples that represent your research question well or effectively.
The basic research questions (i.e., when, where, why, how, and whom) and how these influence sampling.
· First, when will observation, a labor- intensive endeavor, be possible? you must be realistic about what is possible, so your own availability and that of those you wish to observe can be balanced.
· Second, where will you go? We must also decide which sites (Question 2) are likely to yield answers to our research questions.
· When you are in the field observing, what types of phenomena do you want to see, and how are these phenomena likely to manifest themselves? You will have to sample from more opportunities than you can possibly follow up, so you have to think of how to use your time effectively as you choose when and what to witness.
· Finally, you need to consider whom you want to observe. you must decide who is most interesting in order to answer your research question.
One final way to think about sampling in observational designs is to return to the continuum of question types used in Chapters 4 and 8: (1) descriptive; (2) exploratory; (3) explanatory; (4) theory testing; (5) and theory building.
· First, there are research questions that require thick description of a phenomenon, and sampling decisions will seek to balance resource concerns with educated guesses about how much descriptive detail will be enough.
· Next are those exploratory research questions that imply elaboration on prior thick descriptions where questions are more specific and in which we explore variations we are coming to understand.
· Third, we come to “why” and “how” questions that seek to explain phenomena.
· Fourth are questions that ask us to test prior findings as a means of establishing an emerging theory.
· Finally, we conclude with research questions that seek to refine or build upon a theory and to create the possibility of using the research results to further our scholarship and to shape practice.
Observational research is resource intensive, not only when you are on site observing, but also continuously in the care and feeding of your field notes and transcripts, and then during the coding, analyzing, and interpreting phases of the research. A
Summary:
Observational research designs face distinctive sampling challenges, often related to how opportunities present themselves in the field. Chapter 10 approaches these challenges three different ways: By looking at issues of relevance and representativeness, access, and resources; By considering four basic sampling decisions: when, where, what, and whom; and By considering sampling in light of five types of research questions: descriptive, exploratory, explanatory, theory testing, and theory building.
Most sampling for observations is purposive; you seek cases that are relevant to your research question rather than sampling randomly from a known population. Determining the defensible criteria by which you select your samples is a critical component of your research design
Chapter 16 - Ethics in Observational Research
Key Points and Quotes:
Ethical obligations for observational research include: maintaining the anonymity of the individuals and organizations observed, acquiring informed consent, and safeguarding the study’s ethical dimensions even when observing covertly.
Observational Research vs. Experiments:
The fact that experimental research is the basis of traditional ethical standards makes observational research an outsider, or at least an outlier, in some circles. Another source of difference for observational designs is that relationships are more likely to develop between researchers and the people they study. Finally, a researcher is unlikely to anticipate all the ethical dilemmas that will arise in the field (it is an uncontrolled environment), which places a particular responsibility on the researcher, to be clear about the potential benefits and harms that may result from their decisions and actions.
“You must be prepared as you plan the study to weigh design decisions carefully, judge how engaged you need to be to answer your research question, prepare to articulate your safeguards clearly for a skeptical IRB, and think ahead about what ethical dilemmas may arise in the field as you sample and develop relationships with participants.”
Consent:
· Common practices used in obtaining informed consent in observational research designs include offering some clear benefit to the participants and/or providing them the chance to review your work in some form. To be ethical, you must follow through with any agreements or promises you make and how you will use participant input.
· The more you participate, the more likely you are to need informed consent
· For participant observation, the consent process should describe how you plan to collect and record your observational data.
Harm:
· Rather than causing harm, it is likely that one may observe harmful events.
· One must refrain from exploiting the vulnerabilities of a population.
· Tangible and intangible losses must be considered and prevented by determining what is really necessary to address your research question.
· If you learn something that could harm someone during data collection you can 1) address the problem at the site 2) withdraw from the site or 3) stop the study.
· Harm to participants can occur after the study (like Humphrey’s experiment)
o When potential harm is recognized the researcher has to weight the harm.
o Consider using triangulated data sources to elaborate observations.
o Member checking and ensuring privacy and security are still relevant in the analysis and reporting phase.
Privacy:
· Should be considered in the design phase. A researcher should not ask more than they need.
· The principal approach to maintain privacy during data collection is prevention. Use coded identifiers whenever possible.
Summary:
Because naturalistic and participant observations cannot be fully planned in advance there will always be considerable uncertainty about the ethical implications of a researcher’s actions. One must approach the study with a sense of seriousness and sensibility.