3 Analyzing Research & Writing the Literature Review

Writing as a Social Work Skill 

Peer-reviewed journals are considered our most trustworthy form of research-based data and knowledge. During peer review, two or more experts with knowledge of the social issue being studied or research methods being used evaluate the quality of the study; they review the study manuscript for issue development, methodological considerations, and strong theoretical arguments. Only after an iterative process of revisions will the study be published for a broader audience. Most evidence presented in your paper should incorporate peer reviewed literature.

There are five primary types of articles that fall under peer reviewed literature: Reviews (i.e., Meta-analysis or Systematic Review), Quantitative, Qualitative, Mixed Methods, or Theoretical. Peer reviewed articles are arranged into a hierarchy; those studies which have more robust research designs, increase our faith in the findings and are more generalizable fall into level 1; articles with more limited designs follow into lower levels on the hierarchy (see diagram below). In the pages that follow, we will briefly define each of these article types.  

Reviews include meta-analysis or systematic reviews and are at level 1 in the hierarchy. Reviews are a great place to begin developing your literature portfolio (more on this later), as they combine raw data from multiple studies and analyze the pooled data using tested methods. They often bring together at least a decade’s worth of knowledge about a certain issue, use tried and proven methods for collecting and analyzing the data, and present interpretations. 

There are several frameworks for completing a scoping literature review, as identified here. The most common methodology you will encounter is the PRISM method. This has been shown to be the gold standard in meta-analysis across the field and identifies for the reader the strength of evidence before them. Sampling within the review will look different than how we commonly think of sampling (as you’ll see below), in that analyzing the sampling plan of a review is not about the human participants, but the journal articles (studies) chosen for analysis. The key to remember related to review sampling is to think critically regarding the total number of studies identified, then the total number of studies included, contextualized through the sampling frame (e.g., criteria for study inclusion). For example, if the authors state that they identified 200 articles but only included three, I would view the findings with more skepticism than if they had included 60 studies.

Finally, in a review, data collection methods are determined by the measures used in the included studies. Authors should clearly indicate whether these studies employed reliable and valid measures. It is common for some studies to meet these standards while others do not, so pay close attention to the authors’ justifications to help clarify potential limitations. Overall, reviews offer a broad understanding of theory and strengthen the ability to generalize findings to wider populations.

Quantitative

Quantitative Articles (Levels 2–4)

Quantitative research is objective in nature, relying on the systematic collection of empirical, numerical data. Through measurement and statistical analysis, it tests hypotheses about general laws that apply to human behavior. In the hierarchy of evidence, quantitative studies typically fall within Levels 2–4 (comparison chart [PDF]). Most often, they are outcome evaluations—studies assessing the effectiveness of an intervention for specific populations on a targeted change variable (e.g., depression, anxiety, grief).

Level 2: Evidence-Based Practice (EBP) & Experimental Designs

Level 2 studies use rigorous designs that provide strong evidence for practice.

• Evidence-Based Practices (EBPs) are modeled after the medical field, often using randomized controlled trials. According to Drisko & Grady (2015), EBPs integrate four equally weighted components:

  • Current client needs and situation

  • The most relevant research evidence

  • Client values and preferences

  • Clinician expertise

These studies typically include multiple populations in multiple settings and aim to prove an intervention’s effectiveness for certification by entities such as the Substance Abuse & Mental Health Services Administration (SAMHSA) (EBP resource search).

• Experimental Designs share similarities with EBPs but may not be double-blind. In these studies, researchers know which participants are receiving the intervention and which are not (placebo). The primary goal is to determine whether a causal relationship exists between variables. Key elements include:

  • At least two conditions (intervention and control group)

  • Random assignment, meaning each participant has an equal chance of being placed in any condition

Level 3: Quasi-Experimental Designs

Quasi-experimental studies also include at least two conditions but do not involve random assignment. Control groups are used, but participants are placed into groups through non-random means.

Common in social work and other social sciences, these designs often avoid randomization for ethical reasons—such as not denying services to individuals in need. While waitlist controls can be used, the ethical priority is to maintain fairness, such as “first come, first served” practices.

Level 4: Pre-Experimental Designs

Pre-experimental studies are widely used in the Evaluation Capstone option and include designs such as:

• Case studies

• Needs assessments

• Correlational studies

• Customer satisfaction surveys

• Qualitative studies (also Level 4; see qualitative section below)

These studies do not use randomization or control groups, though comparison groups may be present. One-group pre–post outcome evaluations are common but have limited external validity. Demographic comparisons (e.g., by socioeconomic status, gender, race) may be included, providing insight into the study population but not generalizable to other communities.

Level 5: Theoretical and Anecdotal Sources

At the lowest level of the hierarchy, Level 5 includes theoretical articles, anecdotal evidence, and other sources that offer valuable insights but lack empirical testing

Identifying Relevant Research 

Okay, now that you have a sense of the types of sources you will be reviewing to help you develop your literature review, it’s time to start developing a portfolio of resources. The library, search engines, and AI systems like Research Rabbit are important tools in this process. One way to begin is to develop a list of key words based on the issue you want to explore. While this list may be the simplest way to guide your search, it may also result in an overwhelming number of studies. If you are overwhelmed by the amount of information, you can limit the returns by using key words such as, peer reviewed journals or meta-analysis, identify a specific geographic location, or setting time boundaries (i.e. 2015 – present). At times, I will also put a limit on the number of articles that get me started. Such that, I set a boundary of pulling no more than 7 articles as a starting point, and once I’ve hit that limit, I stop my search and begin a review of what I’ve pulled.

The second method of identifying relevant research is through the development of guiding questions. Guiding questions are those questions you want to ask of our knowledge base. Why does the issue exist? What interventions are most effective for addressing this issue? Who is most impacted by the issue? Guiding questions may help to bound your search more than keywords, thereby streamlining your process. You may consider using the 5 Why’s method described earlier, particularly if you’d like to do a deeper dive into the root causes of your chosen issue. Both keywords and guiding questions can be used through your university library and most search engines.

Finally, consider the use of AI in beginning your search. You can put a guiding question into ChatGPT or a similar app and receive an overview with a list of references to get you started. Systems like Research Rabbit allow you to ask a guiding question or upload an article you’ve located, and it will draw you a concept map that demonstrates the connections between your identified article and other similar articles. This tool will provide information on how often the article has been cited, which connections are most significant, and those that appear to be the most significant to your understanding. While this guidebook will not provide full directions on utilizing AI systems, you can find additional information and resources in this AI Statement and Resources document.  

The Theory Based Literature Review 

To write a comprehensive literature review, you will need a diverse body of knowledge to work from. The two key elements of your literature review that come from our knowledge are the theory of the problem/issue and the theory of change. This should be your clue that you are writing a theoretical literature review. A theory is “a systematic set of interrelated statements intended to explain some aspect of social life” (Rubin & Babbie, 2017, p. 615). A theory allows us to explain and answer the why questions. Why are these two things connected? Why is there a relationship between these experiences? Theory is the answer to most why questions. For example, it’s widely believed that a healthy breakfast for children is important for their engagement in education and ability to learn. Great, but why do we believe this? Because researchers have found that food is fuel, it gives us the energy and calories we need to engage with the world.  Remember that when talking theory, it’s not important that you introduce a popular theory that books are written about (e.g., attachment, strengths based, trauma), we have no name for the theory presented above. Therefore, what is important is that you describe why we believe something is reliable, valid, and/or trustworthy, and not just that it is. When writing your literature review, think about telling a story. A story that has a central idea with a beginning (theory of the problem), middle (the theory of change), and end (new, revised, or confirmed theory).

Theory of the phenomenon

As you begin developing your paper, it is essential to explain why the social issue you are exploring exists and how it impacts individuals and communities at the micro, mezzo, and macro levels. Start by identifying the risk and/or protective factors that increase or decrease the likelihood of experiencing the issue. Then, describe the consequences—what happens as a result of experiencing it.

For example, if your focus is anxiety, the literature might point to trauma, isolation, poverty, or chronic stress as common precursors. Because there is often a long list of potential risk factors, your paper should focus on just three or four. You can acknowledge this limitation by stating, “While there are numerous risk factors, this paper will focus on A, B, and C.” Protective factors are often the inverse of risk factors, so it’s best to choose one direction to avoid repetition.

Remember, this is a theoretical paper—your task is to explain why we understand, for example, that isolation can lead to anxiety. Next, outline the key consequences of the social issue. In the case of anxiety, these might include depression, isolation, substance use, or suicidality. Again, narrow your focus to three or four primary consequences. If you’re unsure which to prioritize, consider your population of interest: “What do clients at my agency most often present with?”

Keep the sequence clear: risks precede the social issue; consequences follow it. Staying focused on the factors most relevant to your field placement will help keep your paper grounded. Finally, recognize that some experiences may function as both a risk and a consequence. For instance, isolation might cause anxiety for one person, but for another, anxiety might lead to isolation. This is where theory becomes critical—it provides the framework and language to explain why certain experiences can occur both before and after the onset of a social issue.

• Preventative approaches target risk factors to stop the social issue from developing.

• Intervention approaches are implemented after the issue exists, aiming to prevent or reduce its consequences.

When discussing the theory of change, address three core questions:

1. What is the intervention, and what does it target? (Provide a brief overview.)

2. Is the intervention effective?

3. Why or why not? (Explain using theory.)

Using theory means explaining why an intervention is or isn’t effective and outlining its pathway for change. For example, if you choose Cognitive Behavioral Therapy (CBT) as an intervention for anxiety, the theory suggests that by helping individuals recognize how thoughts lead to feelings and behaviors, they can interrupt and change behavior patterns. Keep the explanation simple—state why we believe it works.

You are encouraged to explore at least three different interventions. A practical starting point is your field placement: identify which theories or interventions are already being used to address the issue and build from there.

Evidence matters. One study alone is not enough to establish credibility. If you are reviewing CBT, include at least three peer-reviewed articles examining its effectiveness. This principle applies across your literature review—both your theory of the problem and your theory of change should be supported by multiple studies. Research depends on replicability, and so should your analysis. As a general rule, aim for at least three sources to support each major idea.

Theory Development worksheet [fillable PDF]

Critical Analysis of Literature  

An important element of being a good consumer of research is to understand the strengths and limitations that are always present, even within the most structured and bounded project. In addition, the NASW Code of Ethics calls on social workers to stay current and up to date on emerging knowledge, as a means of honoring the value of competence. As researchers and evaluators, we acknowledge that there are always limits to our knowledge and understanding. While we strive to put enough guardrails and protections into place to create a solid understanding; those understandings must be contextualized through the design strengths and limitations. The scientific method is at the core of the analytic process. Below you will read about the classical method of analysis based on a hierarchical model of design, or classic model of analysis. However, as we continue to expand our understanding of the human experience, it is also important that we analyze knowledge from a cultural standpoint as a means of deconstructing the role of white supremacy with research & evaluation, as well as all knowledge construction. Therefore, as a starting point for your learning, a model of multicultural validity is also presented.

Classic analysis, hierarchical model 

Internal & External Validity  

Our goal in reviewing the literature is to present a synthesized, critical analysis of the existing knowledge base. This means not only summarizing what is known, but also evaluating the strengths and limitations of the research as well as any gaps in our understanding. A literature analysis is an essential step in developing as a critical thinker, it demonstrates your ability to assess the quality of evidence and decide how confidently the findings and theories can be used to inform your project.

By learning to evaluate research design, sampling methods, and data collection measures, you strengthen your ability to be a responsible consumer of research and media. This skill not only supports the development of your current project but also contributes to your long-term professional growth.

In this guidebook, we will focus specifically on how to assess a study’s validity by analyzing its design, sampling strategy, and measurement tools (for a more in-depth discussion of these topics, see DeCarlo et al., 2021). At the core of analyzing strengths and limitations is to consider its internal and external validity.

Internal validity refers to the degree to which a study can confidently demonstrate a cause-and-effect relationship between variables. It examines whether changes in the dependent variable (the outcome being measured) can be attributed solely to the independent variable (the intervention or factor being manipulated) rather than to other influences. High internal validity means the study has effectively controlled for confounding variables, biases, or alternative explanations, ensuring that the observed effects are the result of the intervention itself.

Core aspects include:

• Causality: Demonstrating that the independent variable directly influences the dependent variable.

• Control of variables: Reducing the influence of confounders or external factors.

• Accuracy of conclusions: Ensuring observed effects are not due to chance, bias, or methodological flaws

External validity refers to how well research findings can be applied beyond the specific sample or conditions of a study. It addresses whether the conclusions drawn are relevant for different people, settings, or points in time, rather than being limited to the original participants or environment. Studies with strong external validity generate insights that are meaningful and applicable in broader, real-world contexts.

Core aspects include:

• Generalization: Applying results to larger or different populations.

• Real-world relevance: Ensuring the findings have practical use outside the study.

• Transferability: Allowing results to be replicated across various situations or contexts.

While each of these aspects have a role, randomization is one of most rigorous tools for establishing the validity of a study. Randomization is the process of assigning participants to different groups (e.g., intervention and control) purely by chance, giving each participant an equal opportunity to be placed in any group.

Why it’s important:

1. Reduces Selection Bias – Ensures that groups are similar at the start of the study, making it more likely that differences in outcomes are due to the intervention rather than pre-existing differences.
2. Improves Internal Validity – Strengthens the ability to make cause-and-effect conclusions by controlling for confounding variables.
3. Supports External Validity – Increases confidence that the results can be applied to other populations, as the sample is more likely to be representative.
4. Enhances Credibility – Randomized controlled trials (RCTs) are considered one of the highest levels of evidence in research because they combine rigorous control with unbiased group assignment.

In short, randomization helps ensure that your study’s findings are both trustworthy and applicable beyond the specific sample studied.

A Note of Caution: While the research hierarchy can be useful as a heuristic device, it is also problematic in how it values one form of knowledge over others. That is, such hierarchies fail to adequately attend to equity, intersectional identity, and the benefits of a postmodern perspective which values lived experience. We maintain that while all knowledge is not equal, all knowledge can be considered equally within the context of the strengths and limitations of the design. We invite you to think about the hierarchy of evidence, not as a privileged framework, but as a function of the design elements that provide more structure around research methodology, interpretation, and generalizability. The graphic below offers an alternative visualization, and later in this chapter we’ll reference the Multicultural Validity approach to contemporary analysis.

When analyzing qualitative research, we don’t discuss strengths & limitations in the same language, rather we consider trustworthiness. Trustworthiness is a quality reflected in qualitative research that ensures the research is conducted in a credible way; a way that should produce confidence in the findings.  To analyze the trustworthiness of qualitative research, you are looking at four core areas: Credibility, Transferability, Dependability, and Confirmability.  Credibility ensures the findings are believable, trustworthy and accurate and asks the question, how confident am I in the “truth” of these findings? A researcher or evaluator can increase their credibility through several methods such as prolonged engagement, peer debriefing, negative case analysis, & member checking. Transferability & dependability are like the idea of generalizability; it is the extent to which the findings of a study can be applied or transferred to other contexts, settings, or populations beyond the specific study sample, and asks, do these findings have applicability in other contexts? And are the findings consistent & repeatable? Dependability is the extent to which research findings are stable and consistent over time.  Transferability & dependability are strengthened through thick description and an inquiry audit. Finally, confirmability is the degree to which the findings of a study are shaped by the participants and not researcher bias, motivations, and personal interests. Researchers can protect against these limitations through an audit trail, reflexivity and bracketing.

Validity forms the foundation for making accurate judgments about how theory applies to a given issue and determining appropriate actions based on our confidence in shared knowledge (Kirkhart, 2010). Building on this, scholars have emphasized the importance of a multicultural validity model that explicitly incorporates cultural considerations into evaluation and research (Hood, Hopson, & Kirkhart, 2015; Kirkhart, 2005, 2010, 2013). Multicultural validity refers to the trustworthiness of our understandings, decisions, and their consequences when examined across multiple and intersecting dimensions of cultural diversity (Kirkhart, 2010).

This framework highlights five key dimensions, methodological, experiential, relational, theoretical, and consequential, that help identify factors that either strengthen or weaken validity (Kirkhart, 2013). Validity is strengthened when evaluation theory supports the thoughtful selection of epistemologies, methods, and procedures that account for cultural diversity and guard against culturally bound bias. Conversely, it is weakened when culture is overlooked or when diversity is treated through simplistic or stereotypical assumptions (Kirkhart, 2005).

At its core, epistemology, the study of how we know what we know, shapes theory development. It urges practitioners to ground their understanding in the perspectives of those most directly connected to the issue, while also examining their own biases and engaging in reflexive practice to critically evaluate the processes by which knowledge is constructed (Phipps, 2016).

The table below outlines different justifications and threats associated with each domain.

Integrating the Evidence 

The integration section of your literature review functions as its conclusion. Here, you bring together all the knowledge you’ve gathered and provide a final synthesis. Your goal is to highlight the most critical points from your review, focusing on how they justify your project. This section should be well-cited, clearly connecting the evidence to your ideas and conclusions.

Identify any gaps in knowledge or research limitations only if you plan to address them in your project. Highlighting gaps you cannot fill is not an effective strategy, save those ideas for a future directions section in a later semester.

In some cases, your topic may not have significant gaps because it has been extensively researched. That’s okay. In this situation, your role is to replicate existing studies to strengthen the evidence base. While new ideas often receive more attention, replication is essential for building trust in theories and practices. To take this approach, emphasize the strengths in the literature you want to build on, while also acknowledging potential challenges your project may face.

For example, research consistently supports Cognitive Behavioral Therapy (CBT) as an effective treatment. By replicating other studies to test CBT’s effectiveness, whether your findings confirm or challenge the theory, you contribute valuable evidence to the field.

You will end this section with a clear, concise statement of your proposed project, its purpose, and how it is grounded in the literature you have reviewed.