Organizing your instruction is like fitting the pieces of a jigsaw puzzle together.
Use Table 7.1 from your textbook as a reference to complete your response. (Refer to attached document)
Reply to the following discussion, APA style, No AI, plagiarism less than 20 %, 2 or more references.
During clinical rotations at a community health clinic catering primarily to a predominantly Hispanic immigrant population, I faced considerable challenges in promoting preventive care and managing chronic diseases. Many patients exhibited poorly controlled diabetes and hypertension, mainly due to socioeconomic barriers, limited English proficiency, and cultural beliefs regarding health. This experience underscored the complexities that Family Nurse Practitioners encounter when striving to deliver effective health promotion in underserved communities.
In one notable case, a 52-year-old male patient with type 2 diabetes had not engaged with healthcare for over a year. After losing his job during the pandemic and fearing deportation due to his undocumented status, he avoided seeking medical attention. Instead, he relied on traditional remedies rather than prescribed medications, expressing a deep mistrust of Western medicine. This situation highlighted how structural and cultural barriers converge to hinder health promotion efforts.
Health promotion in these contexts requires a culturally competent, trauma-informed, and community-based approach. As a Family Nurse Practitioner, I recognized that establishing trust and demonstrating cultural sensitivity were crucial. I employed motivational interviewing to delve into his health beliefs and collaborated with a bilingual community health worker to bridge any communication gaps. Together, we developed a care plan that integrated some of his traditional practices with evidence-based treatments. This patient-centered approach fostered improved engagement and led to better glycemic control over time.
The literature highlights the necessity of culturally tailored interventions. Vega et al. (2022) indicate that factors such as language barriers, mistrust of healthcare institutions, and lack of insurance significantly contribute to poor health outcomes among Latino immigrants. Family Nurse Practitioners (FNPs) should advocate for systemic changes while also addressing individual barriers through culturally congruent care.
Incorporating social determinants of health (SDOH) into clinical interactions is equally essential. Weir et al. (2021) stress that understanding a patient’s housing situation, food security, and access to transportation can facilitate more relevant and sustainable care planning. FNPs are uniquely positioned to screen for SDOH and connect patients with community resources, thereby extending their impact beyond the clinical environment.
Health literacy is a crucial component in patient care. Many patients in my environment often struggle to comprehend medication instructions and lab results. Implementing practical strategies such as visual aids, teach-back methods, and the use of simplified language has proven effective in significantly enhancing patient understanding and adherence (Thomas et al., 2023).
Moreover, technology presents new opportunities for engagement. Mobile health (mHealth) applications and telehealth platforms have demonstrated their effectiveness in supporting chronic disease management, particularly when they are tailored to meet linguistic and cultural needs (Nguyen et al., 2024). Family Nurse Practitioners (FNPs) can utilize these tools to provide continuous support, especially for patients who encounter challenges in attending frequent in-person visits.
In conclusion, fostering health in diverse populations necessitates a multifaceted and culturally responsive approach. Through advocacy, education, and collaboration, FNPs can play a transformative role in reducing health disparities and empowering vulnerable communities.
Reply to the following discussion, APA style, NO AI, Plagiarism less than 20 %, 2 or more references.
The process of health promotion among various groups of populations is usually challenging in terms of what concerns Family Nurse Practitioners (FNPs), especially where cultural, linguistic, and socioeconomic aspects overlap. When undertaking my clinical studies, I got the chance to collaborate with a group of Somali refugee women, and this demonstrated the extent of obstacles to be faced in health promotion. This reflection touches on the issues that I faced and the culturally responsive solutions that I used to endorse good health practices, mediated by the existing literature.
Challenges in Promoting Health
These women had mostly chronic conditions like hypertension and diabetes type 2 diabetes because they were mostly middle-aged. Nevertheless, there were several obstacles to involvement in preventive care and lifestyle change. Language discordance was one of the most important difficulties. The language used by most of the patients was Somali or Arabic, and most of these patients do not speak English well; hence, communicating health information accurately was a problem. It has been revealed that language barriers are closely linked to adverse health consequences, low patient activity, and a lack of patient satisfaction with the process (Wilandika et al., 2023).
Among other things, there were low levels of health literacy along with language barriers. In cases where the learning resources were translated into Somali, the women found it difficult either to read or comprehend the contents. Western forms of care also did not fit into the traditional concepts of illness and healing, and therefore, it was hard to encourage biomedical care. To illustrate, chronic disease was perceived by some of the patients to be an act of God instead of an issue that could be averted, thus their reluctance to get a routine screening or alter dietary patterns. Lauwers et al. (2024) prove that efficient care of populations is associated with consideration of patient expectations, culture, and decision-making.
Strategies and Interventions
I used a community health worker (CHW) and cultural liaison, who was fluent in Somali, in order to address these obstacles. In unity, we have made alterations in the education process in which oral presentation, visual support, and demonstrations replaced the requirement of using written sources. Culturally appropriate dietary examples and a group discussion enabled patients to ask questions and exchange their views within the atmosphere of support. With this strategy, trust was created and participation was promoted.
Education was performed in groups using the Somali language, and hands-on group demonstrations were provided in self-blood pressure measurement and culturally preferred cooking of healthy meals. As Tiase et al. (2022) stress, FNPs are effective in working with social determinants of health due to their nature of patient-centered interventions, which can acknowledge the language barrier, poor literacy, and cultural barriers.
Conclusion
This clinic experience brought to light the complicated aspects of health promotion among the refugee communities, where cultural beliefs, language, and literacy play a critical role in determining the results of care. Through culturally appropriate and community-based communication strategies and the help of the community health workers, I was capable of establishing rapport and promoting involvement among Somali women. These lessons provide support regarding the need to remain culturally humble, cooperate with local communities, and design flexible teaching strategies in health promotion. We as FNPs must be aware of and provide support with the understanding of the needs of underserved populations based on evidence-based and culturally competent practices.
In the context of health promotion, Family Nurse Practitioners (FNPs) often work with diverse populations that have varied health needs and barriers to care. Reflect on a case or scenario from your clinical experience or studies where you encountered challenges in promoting health within a specific population.
Word limit 500 words. Support your answers with the literature and provide citations and references in APA, 7th ed. format. No AI, Plagiarism less than 20 %, 3 or more references.
Reply to the following discussion, APA style, NO AI, plagiarism less than 20 %, 2 or more references.
The Role of the Nurse Practitioner in Preventive Screening and Intervention to Improve the Health of Young Adults.
The transition to young adulthood represents the most critical phase of human development. In these years, people gain independence, forge their own identities and develop social relationships that are quite different from those of life before college or marriage. But it's also an era fraught with unique health risks. For example, there is growing evidence that this is when young people start to take risk risks: substance abuse, unprotected sex and reckless driving become increasingly common. Counselling may be needed in some cases. While mental health problems and chronic illness are both on the rise among these age groups. – Advanced Practice Nurses (APNs), in particular Nurse Practitioners (NPs), are decisively placed to help mitigate these risks and improve health outcomes Through preventive screening and targeted interventions that are tailored to young adults' needs. Comprehensive Health Assessment and Screening
One of the roles of APNs that has been most effective in dealing with the health problems of young adults is carrying out comprehensive health assessments. The most age-appropriate screenings should always follow. This includes providing regular evaluations for mental disorders such as depression, anxiety and substance abuse, all of which are common enough in this age group We can use screening tools like the PHQ-9 questionnaire on mother-to-best health status during pregnancy and GAD-7 (Generalized Anxiety Disorder Scale) A survey for generalized panic disorder. Second: For alcohol abuse, the APN should use a tool like the AUDIT-C. Drug abuse can be addressed through an interview, which, while private, is not secret. Questionnaires will also have to be given out at times as examples, urine is taken each time someone comes to an APN asking for help because he or she claims to be using drugs. In addition to mental health, preventive screenings need also to cover sexual and reproductive health. APNs must give education on safe sexual practices that screening services are available for STIs (sexually transmitted infections) and also advise about contraception to avoid unwanted pregnancies. For women patients, Pap smears and HPV testing whether or not they are sexually active these days should begin when appropriate; for all patients, regular testing for HIV and other STIs based on risk behavior is also advised.
Health Promotion and Risk Reduction, Counseling
Health education and risk reduction counseling is an essential tool for APNs.
Through a nonjudgmental, culturally sensitive approach, APNs can engage young adults in discussions about their lifestyle choices and future goals.
It can help when someone says they're trying to decide what to do and they aren't sure how to go about it. Using Motivational Interviewing, Which provides a way for patients to explore ambivalence about behavior change in an effective strategy.
By opening the channels of communication, APNs are able to address behavior such as tobacco use, excessive alcohol consumption, recreational drug abuse and what's called "bad eating habits" – all of which have long-term effects on overall health. Injury prevention is another focus that cannot be neglected. As motor vehicle accidents are a leading cause of death in this age group, APNs can advise youth to wear seatbelts and not to drive while distracted or under the influence.
For those living in areas at high risk, conflict resolution and violence prevention must also be the topic of conversation. Chronic disease prevention and management
Beginning in young adulthood, chronic diseases are already beginning to take root. APNs can counsel young adults how to avoid and manage diseases like obesity, hypertension, and type 2 diabetes through routine screenings, lifestyle modification counseling and care coordination. Educating young adults on eating right, being physically active, getting enough sleep and managing the inevitable stress develops a basis for lifelong health. Facilitating Access and Continuity of Care, It is often hard for young adults to get health care. They may not have insurance, be able with limited knowledge about putting in claims or understanding insurance policies, know how to set up an appointment with a provider at any given time when they need one and so forth. There are a number of barriers that have a cascading effect, leading back from past experiences to early adulthood and on into middle age. APNs can help bridge these gaps in care by creating staying power for individuals over time and space, assisting them to find their way through the medical system and by advocating for policies that permit better access. When young adults have established a relationship with their healthcare provider, they can be encouraged to seek regular treatment and to return for follow-up care.
A crucial role in the health of young adults – they provide early detection for risk factors and preventative services to stop them developing, and offer ongoing education. Integrating evidence-based screening tools and intervention strategies – all within a patient-centered environment – can substantially improve an ill person's quality of life; this is especially important in helping lone young people through life crises later on.
Reply to the following discussion, APA style, No AI, 2 or more references, Plagiarism less than 20 %.
The Role of Advanced Practice Nurses in Optimizing Young Adult Health Outcomes
Young adulthood, spanning ages 18 to 26, marks a transition into autonomy, marked by physiological, psychological, and social changes that influence health behaviors and outcomes. During this time, young adults often disengage from pediatric services yet fail to establish relationships with adult healthcare providers, resulting in inconsistent health maintenance. This population faces elevated risks related to mental health, substance use, sexual health, and chronic disease emergence. Advanced Practice Nurses (APNs) are uniquely equipped to fill this healthcare gap by offering comprehensive, developmentally appropriate, and culturally sensitive care focused on both prevention and early intervention.
Young adults face significant public health concerns. According to the CDC (2023), leading causes of death in this group include unintentional injuries, suicide, and drug overdoses many of which are preventable. These issues are often rooted in social determinants of health such as poverty, education, and access to care, and they frequently co-occur with undiagnosed mental illness or risky behavior patterns. Advanced practice nurses, with their strong foundation in both clinical assessment and patient-centered education, play a pivotal role in addressing these complex needs. Their ability to assess psychosocial risks alongside physical symptoms makes them ideal providers for holistic care during this vulnerable period (Thompson & Rivera, 2022).
Preventive care is essential for improving health trajectories in young adults. APNs can implement screening protocols that assess physical, emotional, and behavioral risks. Tools such as the CRAFFT screening for substance use, the GAD-7 for anxiety, and the HEADSS assessment framework provide structured guidance to detect at-risk individuals early (Park et al., 2021). These tools, when used within a trusting provider-patient relationship, facilitate early identification of concerns such as substance misuse, sexual health risks, and mental health struggles often before they escalate into crises.
Health promotion through technology is another key strategy in engaging young adult populations. Mobile applications, patient portals, and telehealth platforms can enhance access to care, improve medication adherence, and facilitate timely communication with providers. A study by Nguyen and Patel (2023) highlights that digital engagement strategies significantly improve health literacy and preventive service uptake among college-aged populations. Advanced practice nurses can use these platforms to provide follow-ups, deliver personalized health education, and monitor chronic conditions or medication compliance in real-time.
Moreover, APNs can help bridge equity gaps by offering culturally competent care that respects the diverse backgrounds of young adult patients. Whether addressing gender identity, language barriers, or economic limitations, APNs are trained to deliver inclusive care that builds trust and promotes long-term engagement with the healthcare system. This inclusivity is especially important given persistent health disparities affecting minority and LGBTQ+ youth.
The long-term benefits of early APN-led interventions are significant. Preventing chronic disease onset, reducing mental health crises, and encouraging healthy lifestyle habits during young adulthood have a compounding effect on population health outcomes and healthcare costs. Studies show that young adults who receive consistent, developmentally appropriate care are more likely to utilize preventive services as older adults and experience fewer hospitalizations (Lee et al., 2022). As primary care providers, APNs can help build a resilient generation through integrated, anticipatory care that supports physical and emotional well-being.
Despite increased abilities across developmental realms, including the maturation of pain systems involving self-regulation and the coordination of affect and cognition, the transition to young adulthood is accompanied by higher rates of mortality, greater engagement in health-damaging behaviors, and an increase in chronic conditions. Rates of motor vehicle fatality and homicide peak during young adulthood, as do mental health problems, substance abuse, unintentional pregnancies, and sexually transmitted infections.
Describe how the advanced practice nurse can play a role in improving the health of young adults through preventive screening and intervention.
Word limit 500 words. Support your answers with the literature and provide citations and references 3 or more in APA, 7th ed. format. plagiarism less than 20 %, No AI
Respond to the following in a minimum of 175 words:
PART 1
Consider the program SNAP that you are improving and draft a memo to your supervisor or board. (https://www.fns.usda.gov/snap/supplemental-nutrition-assistance-program)
PART 2
Draft a 525- to 700-word memo to the stakeholders in which you describe the need, intent, goals, and objectives of the evaluation plan you wish to be implemented.
Provide your statement of purpose. Include your vision, mission, and goals. Answer the following questions:
Provide 1 or 2 examples of the evaluation methods (described in Chapter 8 of the textbook) that you would like to see incorporated.
Cite at least 3 peer-reviewed or similar references to support your assignment.
Format the document according to APA guidelines.
Format your memo according to APA guidelines.
1122924 – SAGE Publications, Inc. (US) ©
articulate, but they influence the way in which we view the world and the choices we make. As with any field, there are particular issues that help to define our ideology. I will refer to these issues as calibrators, taken from multiple definitions of the term. Calibrators divide or mark a scale with gradations to determine the degree of something along that scale. Likewise, calibrators can also be plans that have a specific use or application.
Ideology: a system of beliefs that we use to explain and develop solutions; a philosophy or a way of thinking about a certain topic or issue.
A calibrator, used in the context of evaluation, is a continuum upon which we can consider where our beliefs fall and through which we can apply those beliefs in a particular context. The calibrators presented in this section are not meant to be dichotomous, that is, either-or, but rather a range with the extremes presented as a way to consider where your beliefs might fall along the continuum, as well as the strength with which you hold that belief. Three areas of calibrators are discussed: design calibrators, role calibrators, and methods calibrators. These categories and the calibrators within them are not mutually exclusive, as you will notice some similarities. But, taken as a whole, they should provide you with considerations to shape and focus your own ideology.
Calibrator: a continuum upon which we can consider where our beliefs fall and through which we can apply those beliefs in a particular context.
4.2.1 Design Calibrators
Design calibrators help us to organize our thinking around the overarching research design used in evaluation. Evaluation design is covered in detail in Chapter 8, though considerations regarding how you make design choices will be presented below. It should be noted, however, that other factors often determine what evaluation design we can and cannot use in a specific situation. Thus, regardless of our thinking about these calibrators, it does not necessarily mean that we will be in a position to use any design we choose or make unilateral decisions regarding evaluation design. The design we use will be influenced by the resources, both people and financial, available to the evaluation and the context in which the evaluation is implemented. For instance, in some contexts, the environment might facilitate or even promote stakeholder involvement and foster evaluator access to program participants. In other contexts, the environment might present barriers to evaluation.
Calibrator D1: Design Structure.
At one extreme on the design structure calibrator is the medical model of research. At the other extreme is the anthropological model of research. Important questions include the following:
To what extent can and should evaluation be conducted in a controlled environment, using research designs closely aligned to the medical model of research?
To what extent can and should evaluation be conducted in natural settings, much like research designs used in anthropological research?
In what ways is there a trade-off between causation in controlled settings versus correlation in natural settings?
What is the ideal design structure for program evaluation? To what extent does this design structure vary by the purpose of the evaluation being primarily formative or primarily summative?
Some evaluators believe that unless the underlying research design is of sufficient rigor to make causal conclusions, there is little value in utilizing resources to conduct the evaluation. Other evaluators believe that the controls necessary to implement an experiment based on the medical model create an unrealistic environment within which to evaluate the program, thus limiting the generalizability of findings. There is no doubt that veteran evaluators have already formed ideologies in this area and have strong preferences for design structure in various evaluation environments. Honestly, I understand and respect the arguments at both extremes, and I hesitate to share my ideologies for fear of influencing your own deliberations. However, I will say two things: If the ability to relate a program’s strategies to its goals is compromised, generalizability is a moot issue. Likewise, if the effort for program staff to conduct an evaluation is so cumbersome due to environmental changes necessary to create a controlled environment, evaluation is less likely to occur. Oh—and one more thing—making decisions based on some information is better than decisions made without data (either because evaluation is too cumbersome or not valued), yet even the “some information” needs to be valid and credible. All in all, if I had to take a stand on the design structure, it would be to develop as rigorous of an evaluation as possible (i.e., aim toward the medical model of research), while taking into account stakeholder preferences and contextual constraints.
Calibrator D2: Design Purpose.
Design purpose relates to the process and intent of the evaluation. At one extreme is keeping the program or intervention “pure,” that is, not making any changes to the program during the evaluation. At the other extreme is continuous program improvement, such that the program is adjusted on an ongoing basis throughout the evaluation based on formative data. An argument for the former is that if the program is continuously changing, it is difficult to know what the program really is and the extent to which results are “muddied” by strategies that are not consistently implemented. However, an argument on the other extreme is that it is a missed opportunity, and perhaps even unethical, to not make programmatic improvements that would likely improve results for program participants. Important considerations include the following:
In what ways do mid-evaluation program changes affect the interpretability of findings?
In what ways do mid-evaluation program changes affect the replicability of the program?
To what extent should a program make adjustments during an evaluation based on formative data?
In what ways should formative evaluation be used during an evaluation to improve a program?
1122924 – SAGE Publications, Inc. (US) ©
As with all calibrators described in this section, I understand the arguments for and against the extremes. On many calibrators, like design structure, my preference is a range and highly situational. However, with regard to design purpose, I have a strong preference. You do not need to agree with me, and I hope you will develop your own preferences over time with careful consideration and experience. My preference with regard to design purpose is that program evaluation should focus on continuous improvement. I believe it is one of the features of program evaluation that sets it apart from other forms of research. Evaluation is about improving programs and policies, and I think we have the best chance of doing so if we make continual, deliberate programmatic changes based on data, all while carefully documenting those changes.
4.2.2 Role Calibrators
Role calibrators help us to organize our thinking around the function and responsibility of the evaluator. As mentioned in the above section, the context of the evaluation can influence the extent to which an evaluator can implement an evaluation in the preferred manner. However, the two calibrators discussed below can help shape your own ideology around your preferred role as an evaluator.
Calibrator R1: Evaluator Involvement.
Early evaluation viewed the evaluator as a dispassionate, pietistic expert, brought in to pass judgment. While this may seem harsh, evaluators were not seen as partners, collaborators, or friendly visitors. Evaluators were fairly hands-off when it came to the program. Remnants of this view can still be seen in how evaluator visits are perceived by program staff. Evaluators can make program staff nervous, just as we are nervous anytime we feel evaluated. Even though the evaluation is of a program and not an individual, program staff still have a stake in the findings. If the program is not functioning well, program staff may lose responsibility or even their job. However, more recently, evaluators are often partners with program staff. This partnership can take many forms, from the evaluator working only in an advisory capacity to the evaluator working closely with program staff during all phases of the evaluation. Thus, the evaluator involvement calibrator addresses the role of the evaluator and how the evaluator fits within the program. Important considerations include the following:
What role and relationship should an evaluator have with program staff?
To what extent should the evaluator keep firm boundaries between the evaluation and the program? What are the benefits and drawbacks to keeping such boundaries?
To what extent should the evaluator be a full partner with program staff in determining the direction of the program?
A former colleague of mine liked to refer to an evaluator as a critical friend. Evaluators can be a critical friend, a trusted partner, or an external consultant. The difficulty is determining what the most appropriate role is for any given evaluation.
Calibrator R2: Evaluator Responsibility.
An important consideration with regard to evaluator role is the responsibility the evaluator has to a program and to the organization within which that program is implemented. On the one hand, evaluation can be an external activity, with the evaluator’s responsibility to come in, complete the evaluation, provide a report, and leave. On the other end of the spectrum, the evaluator can view evaluation as a capacity-building activity. In such cases, the evaluator seeks to build processes and facilitate data-driven practices that are still in place when the formal evaluation is complete. Important considerations include the following:
To what extent should an evaluation be external/peripheral to the program?
To what extent should evaluators collect the data necessary for the evaluation without interfering with program processes?
How much should an evaluation try to change program processes to incorporate data collection as an ongoing process?
In what ways can and to what extent should evaluators build structures into programs to facilitate a reliance upon data by program staff post-evaluation?
As with design purpose, evaluator responsibility is a calibrator about which I have strong opinions. You may disagree with my opinion and I encourage you to form your own opinion. However, I will share that I believe evaluators have a responsibility to make a difference, not just in the report or recommendations that they leave behind, but in the systems they create during the evaluation. If, as evaluators, we are truly committed to promoting the use of data to make programmatic decisions, we will work to build capacity within programs such that staff are not dependent upon an external evaluator for data-based decision making. We should facilitate an environment of continuous improvement so that data use remains even as the evaluator moves on.
4.2.3 Methods Calibrators
Methods calibrators help us to organize our thinking around the types of data collection methods we value and how we use the methods to orient our evaluation. The two calibrators discussed below can help shape your own ideology around your views on methods and evaluation focus.
Calibrator M1: Data Collection Methods.
A common debate in evaluation, and in all research, is the value of quantitative and qualitative methods. Many evaluators have a strong preference for one over the other, though I daresay most evaluators recognize the usefulness of employing mixed methods, that is, including both quantitative and qualitative methods in an evaluation. Quantitative methods typically allow evaluators to capture information more quickly from more individuals, and large volumes of quantitative data can be analyzed much more quickly than qualitative data. Quantitative measures are also more reliable, whereas it is much more time-consuming for qualitative researchers to ensure adequate reliability in qualitative data. However, qualitative measures can have adequate reliability with structured and consistent data analysis procedures. An example of quantitative analysis is how the multiple-choice items on your SATs can be scored quickly by a machine, and regardless of how many times they are scored, the results would be the same. In addition, the SAT multiple-choice data from thousands of people can be scored at the same time. On the other hand, analysis of the SAT writing responses is more time-consuming and multiple raters are used to score each essay. Yes, any individual scorer can only score one essay at a time. In addition, these raters must go through extensive training to ensure they are consistent in their scoring. The analysis of qualitative data requires similar techniques that might be used to score an essay and such techniques include inter-rater reliability considerations that are not present in quantitative analysis. However, while quantitative methods allow you to analyze more data, more quickly, and with more reliability, they do not provide the kind of rich detail and description that is inherent in qualitative data. Thus, qualitative data can be used to illuminate quantitative findings, such that we can better understand the meaning behind responses and calculations. Quantitative data can help us to determine the generalizability of findings from qualitative data, by enabling us to create closed-ended items on a topic that can be asked of a larger group of people. Thus, important considerations include the following:
1122924 – SAGE Publications, Inc. (US) ©
To what extent do quantitative methods restrict the ability of evaluators to understand a program’s operation and impact at a deeper level?
To what extent do the smaller samples involved in studies using qualitative methods portray a skewed view of a program? How can samples in qualitative studies be constructed so that findings are representative of a larger stakeholder group?
In what ways can both quantitative and qualitative methods be used to provide both depth and breadth to an evaluation?
Calibrator M2: Methods Focus.
With regard to evaluation methods, another consideration for evaluators is how to focus their evaluation. At the heart of methods focus is whether evaluation designs should be constrained by program goals. Ralph Tyler, first introduced in Chapter 2 as the “Father of Evaluation,” laid the groundwork for what most consider to be the first program evaluation approach: objectives-oriented evaluation. While there were other methods that people used to make decisions prior to focusing on objectives, such as expertise-oriented evaluation based on expert opinion and consumer-oriented evaluation intended to make evaluative judgments for public good, objectives oriented was the first evaluation approach geared toward making a value judgment for a specific stakeholder group (Fitzpatrick, Sanders, & Worthen, 2011). Objectives-oriented evaluation focuses on the goals and objectives of a program. Methods are chosen to measure data based on these objectives, and findings are analyzed to determine the extent to which those objectives were met. On the other end of the methods-focus spectrum is Michael Scriven’s goal-free evaluation. Unlike objectives-oriented evaluation, goal-free evaluation is not designed around the specified goals and objectives of a program. Instead, the stated goals and objectives of a program are viewed as incomplete, potentially biased, and a barrier to fully evaluating the program (Scriven, 1991, 2013). Scriven recommends that evaluators examine the program in its entirety, such that both intended and unintended outcomes are measured. So, while there is some element of measuring objectives in a goal-free evaluation, it is not the sole focus to the extent that additional important outcomes are overlooked. For instance, focusing solely on measuring achievement changes based upon a program to increase the rigor of courses might overlook an increase in the number of students who drop out due to frustration. On the other hand, measuring only a program’s goal of increasing youth participation in summer programs might miss the decrease in neighborhood crime by youth during the summer months. Important considerations regarding the methods-focus calibrator include the following:
Objectives-oriented evaluation: an approach to evaluation where the focus of the evaluation is on how well the program met a set of predetermined objectives.
Goal-free evaluation: an approach to evaluation where the evaluation is not constrained by program goals, but rather focuses on the measurement of outcomes, whether intended or unintended; developed by Michael Scriven.
To what extent are a program’s goals worded as strategies the program intends to implement versus the outcomes that would result if those strategies were implemented as planned? What is an evaluator’s responsibility to work with program staff to truly understand a program’s goals, beyond those stated by the program?
How likely is it that the program will have unintended consequences, either positive or negative?
What is an evaluator’s responsibility with regard to evaluation focus? To what extent is an evaluator only obligated to evaluate the objectives of a program as stated by program leadership? To what extent does an evaluator have a responsibility to study other potential impacts of a program beyond the intended goals?
In the Real World … Revisiting the Cambridge-Somerville Youth Study (CSYS): Ideology. The CSYS was introduced in Chapter 2. The purpose of CSYS (Cabot, 1940) was both to prevent juvenile delinquency among boys as well as to study the effectiveness of juvenile delinquency interventions. It is revisited here to illustrate how ideology relates to design, role, and methods.
While it is impossible to truly know what Cabot’s ideology was with regard to evaluation, we can surmise from descriptions of the study where his beliefs might fall along each calibrator continuum. With regard to the design calibrators, the design indicates he favored the medical model of research design and it does not appear that findings were used for program improvement. With regard to role calibrators, the evaluators were external and do not appear to have had much involvement with the program beyond data collection. There is no evidence that CSYS was a capacity-building evaluation. With regard to methods calibrators, while some qualitative methods may have been used, the predominant methods appear to have been quantitative and focused on the objectives of the CSYS.
What if the CSYS evaluators had had a different ideology? Consider each scenario and identify ways that the change may have affected the findings from the study.
SCENARIO 1: Suppose the evaluators chose to forgo a control group and included all youth in the program.
SCENARIO 2: Suppose findings were used throughout the program to improve services for the children involved.
SCENARIO 3: Suppose the evaluator was someone internal to the program.
SCENARIO 4: Suppose the evaluators worked closely with stakeholders throughout the program, building processes for them to collect and analyze their own data.
SCENARIO 5: Suppose the youth were observed and data were collected from these observations, instead of from instruments designed to measure behavior.
1122924 – SAGE Publications, Inc. (US) ©
SCENARIO 6: Suppose the evaluators did not use the stated objectives of the program to drive the study, but instead examined any potential outcome of the program.
Ideally, the relationship between evaluators and stakeholders would be a partnership, such that decisions regarding the focus of an evaluation can be jointly determined. See “In The Real World” for a discussion of how ideology may have influenced the Cambridge-Somerville Youth Study.
4.2.4 Calibrators and Ideology
The six calibrators discussed above are provided for you as areas to reflect upon as you develop your own ideology around evaluation. Figure 4.2 includes a graphical representation of how these calibrators shape ideology—and how ideology, in turn, guides our choices regarding evaluation designs and approaches. On the right side of the diagram are additional influencers that affect our use of evaluation designs and approaches. For instance, resources and context can constrain the types of research designs that might be employed. Evaluator skills and experiences, as well as the degree of access we have to stakeholders, influence the approaches that we are able to take with regard to a particular evaluation. The following two sections will address evaluation design and approaches. The section on designs focuses on how they were shaped by early evaluators and only includes a brief explanation of their purpose (Chapter 8 provides detailed information on evaluation design). Evaluation approaches describe some common evaluation approaches in the field and who contributed to the development of each approach.
Description
Figure 4.2 Evaluation Ideology Calibrators and Influencers
Quick Check 1. How does an evaluator’s ideology affect their choice of evaluation designs and approaches? 2. Do you think an evaluator should build evaluation capacity among the staff of the program they are evaluating? Why or why not? 3. If you had limited resources for an evaluation, would you use qualitative or quantitative methods? Explain your reasoning. 4. Explain the six calibrators that affect an evaluator’s ideology. What are your thoughts on each calibrator? Do you have strong preferences regarding
any of the calibrators?
4.3 EVALUATION DESIGN Ideology influences the evaluation designs we choose to use. In particular, our philosophy regarding the design and methods calibrators drives the overall structure of our evaluation. While there are additional factors that affect and constrain choices regarding evaluation design and methods, our underlying ideology shapes the extent to which we view different research designs as strong or weak. In this section, major contributors to evaluation design will be discussed, within the framework of the evaluation designs themselves. Chapter 8 will explore evaluation design in more detail.
4.3.1 Experimental Designs
Donald Campbell.
Donald Campbell was one of the most critical pioneers in the call for social experimentation in the field of evaluation (Rossi, Lipsey, & Henry, 2018). His groundbreaking work pioneered the application of the experimental model used in psychological research to the evaluation field (Christie & Alkin, 2013). The
1122924 – SAGE Publications, Inc. (US) ©
experimental model of research includes random assignment of subjects to a program/intervention or to a control condition. Campbell’s perspective was that decisions about policy and programs should be made on the basis of experimental research. His perspectives are detailed in his 1969 article “Reforms as Experiments.” For over half a century, his work has guided social science researchers and evaluators on how to conduct rigorous research aimed at establishing causal inference. Causal inference is the ability of evaluators to claim that the program they are evaluating is responsible for the outcomes they measured. One of the most influential books in the field was written by Campbell and his coauthor Julian Stanley. Campbell and Stanley’s (1963) seminal work Experimental and Quasi-Experimental Designs for Research is one that every researcher and evaluator should have in their library. They detail design considerations for randomized controlled experiments, quasi-experiments, and nonexperimental studies. Their work has had a lasting impact on the field of evaluation and has facilitated the use of both experimental and quasi-experimental designs (Shadish & Luellen, 2013). It began a shift in the field, which led to randomized experiments being considered the “gold standard” design for establishing causal inference (Christie & Alkin, 2013).
Causal inference: the ability of an evaluator to claim that the program they are evaluating is responsible for the outcomes they measured; causality can be claimed with experimental designs.
Robert Boruch.
Similar to Campbell’s legacy, Robert Boruch has been instrumental in furthering the use of randomized experiments in the evaluation field. One of Boruch’s (1997) most influential works, Randomized Experiments for Planning and Evaluation, provides a practical guide to randomized experiments for evaluators. Boruch is a strong proponent of using randomized experiments, promotes them as the most effective method for evaluating a program’s effects, and argues that any program can employ randomized experiments to determine their effectiveness. As stated by Christie and Alkin (2013),
,
1122924 – SAGE Publications, Inc. (US) ©
evaluation design will still yield useful information. Yet a strong logic model within a rigorous evaluation design will enable much stronger conclusions regarding program effectiveness and impact. As you have likely surmised, a weak logic model within a strong evaluation design provides little useful information, just as an unreadable treasure map within a sturdy home brings you no closer to the treasure. That said, in this section you will add strength and depth to your logic model by continuing to build upon the evaluation matrix you began in Chapter 7. Methods and tools will be identified or developed for each indicator on your logic model, addressing the question, How will you collect your data?
Although there are many evaluation methods, most are classified as qualitative, quantitative, or both. Qualitative methods rely primarily on noncategorical, free response, observational, or narrative descriptions of a program, collected through methods such as open-ended survey items, interviews, or observations. Quantitative methods, on the other hand, rely primarily on discrete categories, such as counts, numbers, and multiple-choice responses. Qualitative and quantitative methods reinforce each other in an evaluation, as qualitative data can help to describe, illuminate, and provide a depth of understanding to quantitative findings. For this reason, you may want to choose an evaluation design that includes a combination of qualitative and quantitative methods, commonly referred to as mixed methods. Some common evaluation methods are discussed below and include assessments and tests, surveys and questionnaires, interviews and focus groups, observations, existing data, portfolios, and case studies. Rubrics are also included as an evaluation tool that is often used to score, categorize, or code interviews, observations, portfolios, qualitative assessments, and case studies.
Qualitative methods: evaluation methods that rely on noncategorical data and free response, observational, or narrative descriptions.
Quantitative methods: evaluation methods that rely on categorical or numerical data.
Mixed methods: evaluation methods that rely on both quantitative and qualitative data.
Before delving in to different methods, it is worth mentioning the ways in which the terms assessment and survey are sometimes used and misused. First, while the term “survey” is sometimes used synonymously with “evaluation,” evaluation does not mean survey. A survey is a tool that can be used in an evaluation and it is perhaps one of the most common tools used in evaluation, but it is just one tool nonetheless.
Another terminology confusion is between “assessment” and “evaluation.” These too are often used interchangeably. However, many in the field of evaluation would argue that assessment has a quantitative connotation, while evaluation can be mixed method.
Similarly, the term “measurement” is often used synonymously with “assessment,” and measurement too has a quantitative connotation. I believe the confusion lies in the terms “assess,” “evaluate,” and “measure”; they are synonyms. So, it only makes sense that assessment and evaluation, and sometimes measurement, are used synonymously. And while there is nothing inherently wrong with using these terms interchangeably, it is a good idea to ask for clarification when the terms assessment and measurement are used. Some major funders use the term “assessment plan” to mean “evaluation plan,” but others may use the term assessment as an indication that they would like quantitative measurement. The takeaway from this is to communicate with stakeholders such that the evaluation (or assessment) you design meets their information needs and expectations.
8.2.1 Qualitative Methods
Qualitative methods focus on noncategorical, observational, or narrative data. Evaluation using qualitative methods is primarily inductive, in that data are collected and examined for patterns. These patterns are then used to make generalizations and formulate hypotheses based on these generalizations. Qualitative methods include interviews and focus groups, observations, some types of existing data, portfolios, and case studies. Each method is described in the following paragraphs.
Interviews and focus groups (qualitative) are typically conducted face-to-face or over the phone. We also conduct individual interviews using video conferencing software. Focus groups are group interviews and can also be conducted using video conferencing software, but I have found it is difficult to maintain the richness of discussion found in face-to-face focus groups when conducted using video. However, I have no doubt as we become more skilled with facilitating group discussions where individuals are in varied locations, video focus groups will become an important and invaluable mode of research. The list of interview and focus group questions is referred to as a protocol; an interview protocol can be created with questions to address your specific information needs. The interviewer can use follow-up questions and probes as necessary to clarify responses. However, interviews and focus groups take time to conduct and analyze. Due to the time-consuming nature of interviews and focus groups, sample sizes are typically small, and research costs can be expensive. See Interviews in Qualitative Research (King, Horrocks, & Brooks, 2018) and Focus Groups (Krueger & Casey, 2014) for more information on designing and conducting interviews and focus groups.
Observations (usually qualitative but can be quantitative) can be used to collect information about people’s behavior, such as teacher’s classroom instruction or students’ active engagement. Observations can be scored using a rubric or through theme-based analyses, and multiple observations are necessary to ensure that findings are grounded. Because of this, observational techniques tend to be time-consuming and expensive, but can provide an extremely rich description of program implementation. See the observation section of Robert Wood Johnson Foundation’s Qualitative Research Guidelines Project (Cohen & Crabtree, 2006) for more information and a list of resources on using observation in research.
Existing data (usually quantitative but can be qualitative) are often overlooked but can be an excellent and readily available source of evaluation information. Using existing data such as school records (e.g., student grades, test scores, graduation rate, truancy data, and behavioral infractions), work samples, and lesson plans, as well as documentation regarding school or district policy and procedures, minimizes the data collection burden. However, despite the availability and convenience, you should critically examine the quality of existing data and whether they meet your evaluation needs.
Portfolios (typically qualitative) are collections of work samples and can be used to examine the progress of the program’s participants throughout the program’s operation. Work samples from before (pre) and after (post) program implementation can be compared and scored using rubrics to measure growth. Portfolios can show tangible and powerful evidence of growth and can be used as concrete examples when reporting program results. However, scoring can be subjective and is highly dependent upon the strength of the rubric and the training of the portfolio scorers; in addition, the use of rubrics in research can be very resource intensive (Herman & Winters, 1994).
Case studies (mostly qualitative but can include quantitative data) are in-depth examinations of a person, group of people, or context. Case studies can include a combination of any of the methods reviewed above. Case studies look at the big picture and investigate the interrelationships among data. For instance, a case study of a school might include interviews with teachers and parents, observations in the classroom, student surveys, student work, and test scores. Combining many methods into a case study can provide a rich picture of how a program is used, where a program might be improved, and any variation in findings from using different methods. Using multiple, mixed methods in an evaluation allows for a deeper understanding of a program, as well as a more accurate picture of how a program operates and its successes. See Yin (2017) for more information on case study research.
1122924 – SAGE Publications, Inc. (US) ©
8.2.2 Quantitative Methods
Quantitative methods focus on categorical or numerical data. Evaluation based on quantitative data is primarily deductive, in that it begins with a hypothesis and uses the data to make specific conclusions. Quantitative methods include assessments and tests, as well as surveys and questionnaires, and some types of existing data. Each method is described in the following paragraphs.
Assessments and tests (typically quantitative but can include qualitative items) are often used prior to program implementation (pre) and again at program completion (post), or at various times during program implementation, to assess program progress and results. Assessments are also referred to as tests or instruments. Results of assessments are usually objective, and multiple items can be used in combination to create a subscale, often providing a more reliable estimate than any single item (see Wright, 2007). If a program is intended to decrease depression or improve self-confidence, you will likely want to use an existing assessment that measures depression or self-confidence. If you want to measure knowledge of organizational policies, you may decide to create a test based on the policies specific to the organization. However, before using assessment or test data, you should be sure that the assessment adequately addresses what the program intends to achieve. You would not want the success or failure of the program to be determined by an assessment that does not accurately measure the program’s outcomes.
The reliability and validity of an instrument are important considerations when selecting and using instruments such as assessments and tests (as well surveys and questionnaires). Reliability is the consistency with which an instrument measures whatever it intends to measure. There are three common types of reliability: internal consistency reliability, test–retest reliability, and inter-rater reliability. See Figure 8.2 for a description of each type of reliability.
Reliability: the consistency with which an instrument measures something.
Validity is the accuracy with which an instrument measures a construct. The construct might be anxiety, aptitude, achievement, alcoholism, or self-confidence. There are four types of validity: content validity, construct validity, criterion-related validity, and consequential validity. See Figure 8.2 for more information on each type of validity.
Validity: the accuracy with which an instrument measures a construct.
1122924 – SAGE Publications, Inc. (US) ©
1122924 – SAGE Publications, Inc. (US) ©
Figure 8.2 Reliability and Validity
When choosing an assessment or creating your own instrument, you should investigate the technical qualities of reliability and validity to be sure the test is consistent in its measurement and to verify that it does indeed measure what you need to measure. Further, taking a subset of items from a validated instrument to create a new instrument does in fact create a new instrument, with untested reliability and validity. Results from an instrument that is not valid are, in turn, not valid. That is, using an instrument that has not been validated through the examination of reliability and validity can result in erroneous and costly decisions being made based upon those data.
Surveys and questionnaires (typically quantitative but can include qualitative items) are often used to collect information from large numbers of respondents. They can be administered online, on paper, in person, or over the phone. In order for surveys to provide useful information, the questions must be worded clearly and succinctly. Survey items can be open-ended or closed-ended.
Open-ended survey items allow respondents to provide free-form responses to questions and are typically scored using a rubric. A rubric is a scoring guide used to categorize text-based or observational information based upon set criteria or elements of performance. See Figure 8.3 for more information on rubrics. Closed-ended items give the respondent a choice of responses, often on a scale from 1 to 4 or 1 to 5. Surveys can be quickly administered, are usually easy to analyze, and can be adapted to fit specific situations.
Rubric: a guideline that can be used objectively to examine subjective data.
Building a survey in conjunction with other methods and tools can help you to understand your findings better. For instance, designing a survey to explore findings from
