Notes from Week 4 Reading (“Analyzing Quantitative Data”)
Using illustrative, extended examples, the chapter “Analyzing Quantitative Data” focuses on defining the framework for “good” research – that is scholarship focused on the accurate representation of statistical/quantitative data.
This chapter discusses hypothesis testing involving numerical averages. It also manages the complexity of statistical calculations by discussing the principles that underlie mathematical formulas (rather than the formulas per se).
When research that draws conclusions or inferences about a general subject or group is based on measurements taken from a smaller sample, two standards must be met: the validity of the measurements and the reliability of the inference.
When designing for external validity, ensure that the sample group itself is a fair representation of the general population of interest.
The issue of quantitative reliability is one of statistical significance.
Some basics of statistics: 1) there are types of statistical calculations that we run before tests, statistical tests that are tests, and statistical tests that are follow-up...Not all statistical tests perform the same function; 2) each test can only be performed on only a certain type of data; 3) a test is typically named after a letter that appears in its formula or the name of the person who created it; and 4) to determine the statistical test to use, you have to determine the form your data are in and then lookup that data in a table that describes appropriate statistical tests.
Two principles underlie all statistical formulas: 1) the smaller the variable in the data, the more reliable the inference and 2) the bigger the sample size the more reliable the inference.
Standard deviation is represented by SD and sample size is represented by n.
Tools for quantitative analysis include: Excel, Jamovi, and SPSS (Statistical Package for the Social Sciences).
Some Key Terms/Concepts
Dependent Variable – the measurable outcome that the researcher uses to gauge the effect of changing the independent variable.
Independent Variable – the condition that the researcher deliberately manipulates. Also referred to as the Intervention – an action taken to manipulate a variable.
Quantitative Data – data that can be expressed in numbers. (Can also be used in descriptions like age, salary, and education level.)
Operationalizing – defining a concept so that it can be measured.
Internal Validity addresses the question “did you measure the concept that you wanted to study?”
External Validity addresses the question “did what you measure in the test environment reflect what would be found in the real world?” External validity can be managed by taking care when you set up the test that the conditions you create in your test environment match the conditions in the general environment as much as possible.
Reliability refers to the likelihood that the results would be the same if the study were repeated. Reliable research tries to ensure that any differences detected are actually the result of the object of the study and not caused by differences in the samples or by random variation.
Statistical significance – a measure of the degree to which chance could have influenced the outcome of a test.
Descriptive statistics – describe a specific type of data while inferential statistics make inferences about a larger population based on the sample data.
Random selection means that selection and assignment of each test participant is independent and equal (a contrast to problematic samples of convenience which utilize participants on hand). “Independent” means that selecting one participant to be in a particular group does not influence the selection of the other. “Equal” means that every member of the population has an equal chance of being selected.
Hypothesis represents the assumption a researcher is trying to make. It should clearly state the independent variable, dependent variable, and the direction (if any) of the hypothesized effect.
A census asks every person in a target group (contrast to a survey which represents a sample).
The tails argument refers to a one-tailed or two-tailed test. Researchers use a one-tailed test (aka directional test) if the hypothesis is describing an intervention that you believe will cause a difference in a particular direction. In a two-tailed (non-directional) test, the test hypothesis would be supported by a difference in either direction.
Correlation looks at the relationship that exists between two variables. It does not mean causation.
Types of Statistical Analysis: ANOVA (Analysis of Variance) is a type of analysis that enables you to compare the means of three or more independent variables. Regression analysis is another type of statistical analysis which interpret the strength of correlation. Chi squared analysis is used when a variable of interest has been operationalized as a percentage.
“You aren’t supposed to memorize all of these different statistical tests and formulas.” (61)
“..you can ‘trust’ the outcome from statistical formulas more when the data in the sample do not vary much. The more the data vary, the lower the reliability of inference.” (63)
“The external validity of the study is dependent on how realistically the profiles of the test participants match the profile of the population of interest. The reliability of the study is affected by the size of the sample.” (64)
“It is much more difficult to prove that a thing is true than it is to prove that it is not true.” (68)
“Remember that as you design quantitative methods in your research, you should always seek to produce research that is RAD – replicable, aggregable and data-supported.” (88)
Strengths and Weaknesses
“Analyzing Quantitative Data” is refreshingly accessible in its explanation of terms. The chapter provides detailed examples with accompanying graphs that provide a helpful framework for introductory researchers. As stated previously, my future research will most likely utilize archival or observational research and not statistical analyses. However, the author provides a useful understanding of the complexities associated with responsible research studies.
What connections, if any, can you make to other authors?
The author begins the chapter by asserting that when “research involves human subjects, consider your ethical obligations to your participants by establishing informed consent.” In addition to tying to the standards of the IRB, this imperative is detailed in the writings of Vogt et al in When to Use What Research Design and Selecting the Right Analyses for Your Data. Although the Vogt publications explore ethics in detail, this chapter centers the importance of consent as a baseline for the construction of effective research studies.