Hot TopicsResearch & Recommendations

Understanding the Design Behind the Study 

Posted SMAHRT Research

By: Ethan Yu

When reading a research paper, many people accept the results at face value without considering how the study was designed. However, even highly cited studies can have basic flaws such as inappropriate control groups, biased sampling, small sample sizes, or improper statistical analyses. Evaluating the logic and writing of a paper is important, but doing more to analyze the structure is essential for maintaining accountability in the scientific community. Understanding study design allows both readers and researchers to evaluate whether the information is trustworthy and meaningful.   

Understanding Study Design 

Study design is a huge aspect of evaluating a research paper. In general, study designs fall into two broad categories: observational and experimental studies. Observational studies happen when researchers observe what is happening, with minimal interference. When the data is collected with this zero to little interference, researchers can confidently show associations and correlations, but are unable to infer a cause-and-effect relationship. However, this lack of interaction allows for the observational data to be more easily collected. Experimental studies happen when researchers actively change a variable and control the environment to see what happens. These types of studies can show a cause-and-effect relationship, but often are more expensive and time-consuming than observational studies. Both designs play essential roles in scientific research, each with its own strengths, limitations, and applications. 

Observational Studies 

Observational studies are a type of research where researchers observe and analyze outcomes without interfering with any of the variables. For example, a researcher might study the relationship between screen time and sleep quality in teenagers by analyzing survey responses without changing any behaviors. By collecting information that already exists, investigators can identify patterns, detect associations, and explore potential risk factors in real-world settings. Although observational studies cannot establish a cause-and-effect relationship, they are valuable for creating hypotheses, understanding prevalence, and more.   

Cross-sectional vs Longitudinal   

Observational studies can be grouped in a various ways, such as by time. When grouping by time, observational studies are normally defined as cross-sectional or longitudinal. Cross-sectional observational studies are conducted with information or data from a brief period of time. On the other hand, longitudinal observational studies are conducted with information or data collected from an extended period of time, with multiple time points of collection. For example, a cross-sectional study may survey responses from teenagers about their social media use on a certain day, while a longitudinal one may track a group of teenagers over several years. Cross-sectional studies are useful for assessing prevalence or associations, while longitudinal studies are better for looking at trends over time.  

Experimental Studies   

Experimental studies take a different approach to research by actively changing one or more variables to observe their effects on outcomes. For example, a researcher might test whether taking breaks from social media improves mood by asking one group of participants to stop using social media for one week while another group continues as usual. By controlling both the environment and the variables involved, researchers can more effectively test for cause-and-effect relationships. The two most common types of experimental designs are randomized controlled trials and quasi-experimental studies.   

Randomized Controlled Trials   

Randomized controlled trials involve randomly assigning participants to a control and treatment group, minimizing confounding and bias through randomization. Confounding happens when an outside factor influences the results. Bias refers to problems in how the study is designed that can unfairly affect the outcome. Both can make it hard to trust findings if not properly addressed. Although randomized controlled trials can be expensive and time-consuming, they are considered by researchers to be the gold standard in experimental design and causal evidence. For example, randomized controlled trials can be seen in clinical trials used to test new antidepressants, where a treated group is compared to a control group.   

Quasi-Experimental Design 

Quasi-experimental design studies are often more feasible than randomized controlled trials because they do not require full randomization of group assignments, yet still involve changing a variable of interest. Although this lack of randomization weakens the evidence of causal relationships, quasi-experimental designs are commonly used in real-world settings where experimental control is not possible. For example, a quasi-experimental design could be used to evaluate the effectiveness of removing cellphones from schools by implementing new policies at School A but not School B. Participants were not randomly assigned to their schools, making this study a quasi-experimental design.   

Past Designs in SMAHRT  

Our team here at SMAHRT has used a wide variety of study designs, including observational designs such as cross-sectional and longitudinal studies, as well as experimental studies. Many of our study designs have been observational, allowing us to explore patterns and associations between technological behaviors and mental health outcomes. We have also used longitudinal designs to explore changing patterns over time, allowing us to understand trends and how habits evolve over time. Examples of such paper includes our “Using Media to Understand Mechanisms of Behavior Change” study in 2011. On the experimental side, SMAHRT has conducted quasi-experimental designs, such as comparing outcomes between unrandomized groups that started using different digital health interventions. In some cases where funding is sufficient, SMAHRT has also conducted randomized controlled trials, such as the Social Media and Health Information Study (SMAHI) Phase 2. 

Conclusion 

Understanding the difference between experimental and observational studies is important for critically reviewing research. Each design has its own strengths and limitations, making it crucial to evaluate whether a design has been appropriately used in a study. Whether conducting a study or reviewing a paper, understanding how the research is conducted allows readers to better evaluate the reliability of the work, allowing us to maintain credibility in the research community. Because as a member of a research community, knowing how a study was designed is just as important as what the study found. 

Citations  

Leave a Reply

Your email address will not be published. Required fields are marked *