1  The Scientific Process

How do we gain knowledge?

Imagine the following:

  • Your grandfather says that ‘kids these days spend too much time on the internet.’
  • A Twitter user suggests that teens are becoming dumber because of excess social media use.
  • A peer-reviewed scientific article suggests potential small positive effects of social media use on depressive symptoms.

Who would you trust more? What impacts this decision? How does your own biases impact your decision? What would you suggest to someone asking for advice on monitoring their social media use?

We all have biases, flaws, and are prone to making errors. There were likely numerous times where you and a friend, partner, or family member disagreed about a situation–you saw things differently. Furthermore, our minds are wired to use heuristics to ease the cognitive load of processing vast amounts of information. While it can speed things along for us and do a reasonably effective job in navigating our complex world, it can sometimes lead us astray. As an example, consider someone with a cognitive bias to interpret their friends’ actions in a negative way. This person always believes that they annoy their friends or are a burden to others. Importantly, these thoughts may not be grounded in ‘reality’; the friends may perfectly enjoy the company of the individual in question. While this example may feel extreme, we all exhibit some cognitive biases to some degree in our lives. As a result, and as scientists, it’s imperative to reflect on our biases and how that may impact how we collect and interpret information, and draw conclusions based on the information.

There are many ways to science. Some researchers implement specific methods to counter their potential biases. Others draw on these biases in a reflexive manner, acknowledging that one can never completely separate from their own experiences and biases, and use this to strengthen their understanding of complex topics. Although there are many ‘ways’ to do science, they typically use a systematic approach to generating an argument or idea, and then planning and implementing a method to test the verisimilitude (i.e., truthfulness) of the idea. For the purposes of this course, we will adhere to a commonly employed scientific method in psychology. Namely, null hypothesis significance testing. Very briefly, our research process will consist of:

  1. Generating hypotheses
  2. Designing a study
  3. Collecting data
  4. Analyzing data
  5. Disseminating results
Think about it

Can we be bias free?

What parts of psychology most interest you? Clinical, developmental, social, cognitive, …?

Why do you want to study a specific topic in psychology?

How does your background and, potential, biases impact this decision?

How might these biases impact how we view a topic? For example, how does a view that ‘all suicides can and should be prevented’ impact how someone studies suicide?

1.1 Generating Hypotheses

Before we continue, its important to distinguish some common terms used in psychological research.

1.1.1 Theory

Quite broadly, we start with a theory. A theory is a set of ideas or statements that explain how phenomena–things you observe in the world– work. You encounter theories and apply them all the time. When you throw a ball to your friend, you do so with with an understanding that the Earth is bending space time and will cause the ball to accelerate downwards (i.e., gravity and theory of general relativity). When you go to the grocery store, you believe that you and the other customers are generally good people who have rights and responsibilities that they will abide by (i.e., social contract theory).

A theory is a set of ideas or statements that explain how phenomena–things you observe in the world– work.

There are myriad theories in psychology. In fact, theoretical pluralism is often viewed as a strength and necessity in our field. Human behavior is so complex that we need a diverse set of theories to explain different behaviors in different contexts. For example, consider a theory seeking to explain suicide: the interpersonal psychological theory of suicide (Van Orden, 2010). The following statement can be derived from this theory:

Theory: Thinking about killing oneself is the result of the belief that one is a burden to others. Feeling like a burden indicates that the person takes more from relationships than they provide.

1.1.2 Hypotheses

From a theory, we can derive a hypothesis (or the plural, hypotheses) – a specific statement that predicts something that will happen When we throw the ball, we predict it will come down. When we are shopping for groceries, we predict that we won’t be assaulted or robbed, and that all customers will pay for their goods.

A hypotheses is a specific statement that predicts something that will happen.

Going back to our theory of suicide:

Hypothesis: individuals who are induced with thoughts of burdensomeness (\(x\)) will have more thoughts of suicide (\(y\)) than those who are not induced thoughts of burdensomeness.

Thus:

\(x\rightarrow y\)

Researchers design studies to test hypotheses. Thus, your knowledge of the theory that you are hoping ot test is integral. Hypotheses should be generated from theory.

1.1.2.1 What makes a good hypothesis?

Not all hypotheses are equal. Although not an exhaustive list, there are several features of good hypotheses. These include:

  1. Testable

A good theory can be tested empirically. That is, you can design an experiment that can feasibly collect the data required to test it. Considering the example above using the interpersonal theory of suicide, are we able to design a study to test it? We could randomly assign people to one of two groups: one are provided information that indicate their family and friends believe that they are a burden. The other is provided more general information about their family and friends’ beliefs about them. Then, we can measure people’s suicidal ideation levels after receiving that information. What should we observe based on the above hypothesis?

While this is theoretically possible, it would not be ethically possible. We can not readily try to harm people nor provide harmful information– definitely not with the goal of inducing suicidal ideations. The the research method we use to test a hypothesis should be both practical and ethical.

Think about it.

How might a study look that can ethically test the above theory and hypothesis?

  1. Falsifiability
  • The theory should be able to be proven false. As a simple example, imagine we predict that \(x\) causes \(y\). If we run an experiment and \(x\) occurs, but \(y\) does not, we have evidence that our hypothesis is false.

Indeed, Karl Popper, a famous philosopher of science, proposed that a scientist’s goal should be to prove their theories false. That if we are true advocaes wanting to advance knowledge, that we should bravely try to prove ourselves wrong. The more we can’t, the more evidence that our ideas are correct. When describing Einstein’s theory, Popper writes about the risky nature of Einstein’s experiments:

Now the impressive thing about this case is the risk involved in a prediction of this kind. If observation shows that the predicted effect is definitely absent, then the theory is simply refuted.[Popper, 1962]

  1. Clarity and Precision

Researchers must clearly operationally define the variables of interest. What do we mean by ‘suicidal ideations’ or ‘believing they are a burden to others’? Furthermore, the proposed relationships between variables should be clear. When one increases, do we expect an increase in the other?

A major goal here is for others to know exactly what we are testing or to replicate our findings.

  1. Simplicity

Hypotheses should be as simple as needed, but no more. Thus, hypotheses should offer the simplest explanation possible that does not cross over into radical reductionism. Radical reductionism poses the risk of oversimplifying the complexity of humans. This is often seen in psycho-centric theories that may neglect the socio-cultural and systemic factors that impact our cognition, affect, and behavior.

  1. Theory-derived

Importantly, good hypotheses are grounded in theory or empirical observations. Having ideas and testing them is great. However, it is useful to review the literature to gauge what is known about the topic and what has been researched. It’s not a good feeling to have a great idea, beginning planning, only to find out that it was already done with unpromising results.

1.1.3 Translating Conceptual Hypotheses into Statistical Hypotheses

Translating a word-based hypothesis into a statistical hypothesis is a critical step in psychological research, as it allows researchers to formally test their predictions using statistical methods. A word-based hypothesis, or a conceptual hypothesis, is often a statement that predicts a relationship between two or more variables. For example, a researcher might hypothesize that “higher levels of social support reduce the risk of depression.” While this hypothesis is clear and meaningful in everyday language, it needs to be translated into a form that can be empirically tested, which is where the statistical hypothesis comes in.

Tip

A conceptual hypothesis is often a statement that predicts a relationship between two or more variables

The first step in this translation process involves operationalizing the variables. In the example above, the researcher would need to define how “social support” and “depression” are measured. Social support might be quantified using a standardized scale, and depression might be measured using clinical assessments or self-report questionnaires. Once the variables are operationalized, and under the NHST framework (more to come in a later chapter), the researcher can develop a null hypothesis (\(H_0\)) and an alternative hypothesis (\(H_A\) or \(H_1\)). The null hypothesis typically states that there is no relationship between the variables (e.g., “social support has no effect on depression”), while the alternative hypothesis suggests that there is an effect (e.g., “higher levels of social support reduce depression”).

Next, the researcher chooses the appropriate statistical test based on the operationalization of the variables of interest, nature of the data, and the research question. In this example, they might use a correlation test or a regression analysis to examine the relationship between social support and depression. As some examples (more to come in later chapters):

  • Relationships between two continuous/ratio variables is represented statistically as:
    • \(H_0: \rho=0\)
    • \(H_1: \rho\ne0\)
  • Group differences (e.g., men and women) on a continuous/ratio variables is represented statistically as:
    • \(H_0: \mu_1=\mu_2\)
    • \(H_1: \mu_1\ne\mu_2\)

This process of translating a word-based hypothesis into a statistical framework allows researchers to test their ideas in a statistical way using data that they collect. Additionally, it ensures that psychological research is grounded in empirical evidence and provides clear criteria for evaluating the strength of findings.

1.2 Designing a Study

After we have developed a suitable hypothesis, we can begin to plan out our study. Essentially, we want to develop the research methods to test the hypothesis(ses).

Researchers outline a proposed research plan that typically includes the participants of the study, measures used, and procedure we can expect participants to follow: a Method section. The Method section is essential for explaining how the study was conducted and ensuring it can be replicated.

It typically begins with a Participants subsection. Here, a description of the participants, including relevant demographic details such as age, gender, and ethnicity. The sample size is justified, typically using some form of a power analysis (see a later chapter). Additionally, any inclusion and exclusion criteria used to select participants are detailed (e.g., “we excluded individuals with a diagnosed mental disorder because…”). The participants section also describes how participants were recruited (e.g., through advertisements, schools, or online platforms) and whether any compensation was provided.

Next is the Materials section. The materials and measures used in the study are outlined. This includes any tools or equipment used such as computers or specialized software. Additionally, descriptions of questionnaires, surveys, or psychological tests used to collect data are included here, along with details about their psychometric properties (e.g., reliability and validity). If the study involves specific stimuli (e.g., images, sounds, or videos), these are described as well. For example, if the study included pre-recorded videos showing someone in a fake therapy session, they would be described in detail. Additionally, measures can be provided as online supplements (i.e., people reading your work can easily access the materials, unless there are some copyright or ethical concerns).

Next is the Procedure section. Here, the complete procedure of the study is detailed, which offers a step-by-step account of what participants did during the study. This includes the instructions they received, the tasks they performed, and any experimental conditions they were assigned to (e.g., control or experimental groups). The procedure also details how participants were assigned to these conditions (e.g., random assignment) and the duration of each session. If deception was used, a description of the debriefing process is included.

While these are the core aspects of the method section, it is also common to include other sections.

The analytic plan section may describe in detail the proposed statistical analyses of the research. Part of others being able to replicate your findings means they must have a sound understanding of your exact analytic plan. For example, how (if you did) did you remove outliers? WHy did you choose a specific level of statistical significance? What type of regression and variable entry method did you use?

Additionally, it is typically required to present the ethical approval of your study. This section can briefly explain whether the study received approval from an ethics committee or institutional review board (IRB) (sometimes called a Research Ethics Board [REB]).

1.3 Collecting Data

The next step, after ethical approval, the goal is to carry forward your study. You will collect data until you have reached your a priori sample size (i.e., using your statistical power analysis).

1.4 Analyzing Data

An integral part of research is conducting the appropriate statistical analyses. In essence, we have an hypotheses (i.e., idea/prediction) about how the data should fit together (e.g., \(x\) and \(y\) are correlated; \(x \leftrightarrow y\)). Analyses allow us to model the data (i.e., force some structure to it) to determine how well it fits with our hypotheses. The main goal of this e-text is to outline some commonly employed statistical analyses used in psychological research. As such, the chapters that follow may explain and, through examples, complete statistical analyses. I am currently reworking the book to reduce the amount of R code within the analyses chapters, and instead include analyses in R as standalone chapters at the end.

I try to adhere to this structure in this companion e-text:

  1. State the hypotheses
  2. Set your criteria and analytic plan
  3. Collect data
  4. Analyse
  5. Write your results/conclusions

1.5 Disseminating Results

Importantly, we should communicate to other researchers and the general public exactly what we did, what we found, and what are the practical applications/meanings in an honest and transparent way. This can be through public forums, academic journals, or as registered reports. Ideally, we can accumulate enough evidence to support our theories and, ultimately, how accurately we explain psychological phenomenon.

I strongly recommend reading Chamber’s book The Seven Deadly Sins of Psychology to help uncover some of the darker sides of our current publication structure.

1.6 Concluding Remarks

Through this companion, you will learn about several important considerations. For example:

  • How can we test the hypothesis? Is it testable?
  • What ethical considerations are needed?
  • What is meant by each of the constructs listed above? How can we accurately and reliably measure ‘suicidal thoughts’ and ‘burdensomeness’?
  • Who should we collect data from and how?
  • Once collected, how can I analyse it?
  • How can we share our results?

However, the focus will be on statistical analyses related to the PSYC 2925, 2950, and 3950 course at Grenfell Campus, Memorial University of Newfoundland.

Practice
  1. Identify a theory in psychology you would be interested in testing.
  2. Derive a hypothesis from this theory.
  3. Design a hypothetical study to test the hypotheses.
  • Who are the participants?
  • What materials do you need?
  • What procedure would you follow?
    • Ensure people reading your study design could attempt to directly replicate your results.
  1. What parties would be interested in knowing the results of the study?
  2. How would you communicate your results?