Development and Psychometric Evaluation of the Connectedness with Health Care Providers Scale for Adolescents and Young Adults with Cancer

Introduction

Connectedness with health care providers (HCPs) is an important and meaningful experience for adolescents and young adults (AYAs) with cancer.^1,2 Connectedness is the degree to which AYAs perceive having a close, meaningful, and significant relationship with HCPs; this perception is characterized by positive expressions (i.e., empathy, belonging, caring, respect, and trust) that are both received and reciprocated.³ When AYAs are connected with their HCPs, it fosters a sense of being cared about, understood, and respected as a unique individual. AYAs also feel more confident and comfortable in communicating with HCPs, look forward to interacting with HCPs, are grateful for their HCPs, and experience enhanced well-being during treatment.¹ Contrastingly, when AYAs are disconnected with their HCPs, they are unwilling to communicate with HCPs and experience a sense of anger and resentment toward their HCPs.⁴ Fostering connectedness may have important clinical implications such as AYA participation in decision-making and long-term follow-up.

To foster and maintain connectedness (and prevent disconnectedness), it is imperative to assess an AYA's sense of connectedness, especially during the early phases of treatment. However, there are currently no reliable or valid instruments to measure connectedness with HCPs reported in the literature. Before interventions can be developed to foster and maintain connectedness, there is a critical need to develop an instrument of connectedness.

Purposes of this article are to (1) describe the development of the Connectedness with HCPs Scale (C-HCPS) for AYAs (ages 13 to 21 years); (2) describe its content validity; and (3) report its psychometric properties (dimensionality, internal consistency reliability, and construct validity). Aims of the study included (1) developing an instrument of connectedness with HCPs for AYAs; (2) evaluating the content validity of the instrument through expert panels; (3) assessing the dimensionality of the instrument; (4) evaluating the internal consistency reliability of the instrument; and (5) evaluating the convergent and discriminant validity of the instrument through hypothesis testing.

Aims were carried out in two phases. Phase I involved generating items, having two expert panels (AYAs and clinicians) evaluate the items for content validity, and pretesting the instrument before pilot testing. In phase II, the psychometric properties of the instrument (dimensionality, internal consistency reliability, and convergent/discriminant validity) were evaluated.

Participants were recruited from two pediatric cancer centers in the Midwest. Institutional Review Board (IRB) approval was obtained at both sites before recruiting participants.

Phase I (Aims 1 and 2)

Aim 1: Develop an instrument of connectedness with HCPs for AYAs

The C-HCPS for AYAs is designed to measure the degree to which an AYA perceives that he/she has a close, meaningful, and significant relationship with his/her HCPs. The term, health care providers, refers to doctors and nurses who work in the pediatric oncology setting.

The dimensions and items for the C-HCPS were generated from two studies: (1) a concept analysis of connectedness³ and (2) a phenomenological study of AYA cancer survivors' experiences of connectedness with HCPs.² Nine dimensions of connectedness were identified: intimacy, sense of belonging, empathy, caring, respect, trust, reciprocity, gratitude, and disconnectedness. Two experts involved in the preliminary studies generated items using words from AYAs in the phenomenological study that best represented these dimensions. All the items were worded in a positive way, except for disconnectedness items. A total of 119 items were generated.

Aim 2: Evaluate the content of the instrument through expert panels

Two expert panels (i.e., AYAs and clinician panels) evaluated the content validity of the C-HCPS. Items of the C-HCPS were evaluated based on Lynn's criteria⁵ for establishing content validity. Experts rated each item on a 4-point scale (1 = not relevant; 2 = unable to assess relevance without revision; 3 = relevant, but needs minor revision; and 4 = very relevant). This rating system allowed us to calculate the content validity index (CVI) and determine which items should be retained, revised, or deleted. Based on the AYA expert panel (n = 5), 19 items were deleted. The remaining 100 items were then reviewed by the clinician expert panel (n = 6). Based on the clinician expert panel, 16 additional items were deleted, 9 items were revised, and 9 new items were added. The updated version of the C-HCPS consisted of 93 items presented in a standard questionnaire format, using a 6-point Likert response scale (i.e., 1 = strongly agree, 2 = moderately agree, 3 = slightly agree, 4 = slightly disagree, 5 = moderately disagree, and 6 = strongly disagree). Items for each domain were randomly mixed throughout the C-HCPS.

Our next step was to pretest the C-HCPS in another sample of AYAs (n = 6). We assessed the ease/naturalness of completing the instrument, timing of completion, and clarity of instructions and items. Based on AYA feedback, we made minor adjustments to the instructions (such as adding a question at the end of the questionnaire asking how many HCPs they were thinking of when completing the instrument). All 93 items were retained. The average length of time for completion was 12 minutes.

Phase II (Aims 3–5)

Methods

Sample

Inclusion criteria were (1) currently between the ages of 13 and 28; (2) diagnosed with cancer between 13 and 25 years of age; (3) diagnosed two months prior and no longer than 3 years off treatment; and (4) able to read English. Exclusion criteria were (1) cancers not usually occurring in AYAs, such as breast or prostate cancer, and (2) cognitive impairment.

Sample recruitment

Following IRB approval, a letter was mailed to eligible participants, which described the study and informed the AYAs that someone would be calling to ascertain interest. If the AYAs wanted to participate immediately after receiving the letter, the Principal Investigator's (PI's) phone number and email were provided. An opt-out option (a toll-free number) was provided for AYAs who did not want to be called.

After receiving the letters, 26 AYAs immediately contacted the PI wanting to participate. The PI contacted 77 AYAs through phone—only 3 were not interested in participating (3% refusal rate).

Data collection procedure

AYAs were sent a secure email link to the study website housed in Indiana University's secure REDCap data collection system. The IRB waived the required signed informed consent and parental permission for phase II data collection; thus, after reading the informed consent/assent and acknowledging their consent with a check mark, the AYA was immediately directed to the instruments. Participants' responses were downloaded into a password-protected database for data analysis. No names were associated with instruments.

Instruments

Instruments to evaluate the convergent and discriminant validity of the C-HCPS were hypothesized to have either a significant relationship or nonsignificant relationship with the C-HCPS (Table 1). Instruments hypothesized to have a significant positive relationship with the C-HCPS included (1) the Patient Satisfaction Questionnaire Short Form (PSQ-18)⁶; (2) Perceived Social Support from HCPs⁷; (3) Jalowiec Coping Scale (positive coping subscales)⁸; (4) Herth Hope Index⁹; (5) Reed Self-Transcendence Scale¹⁰; (6) Haase Resilience in Illness Scale¹¹; (7) Index of Well-Being¹²; and (8) Mishel Uncertainty in Illness Scale.^13,14

Instruments hypothesized to have a nonsignificant relationship with the C-CHPS included (9) Experiences of Close Relationships¹⁵; (10) PROMIS^®—Pediatric Anxiety Scale^16,17; (11) PROMIS—Pediatric Depression Scale^16,17; (12) McCorkle Symptom Distress Scale¹⁸; and (13) Jalowiec (defensive coping subscales).⁸

The hypothesized relationships of instruments 2–7 and 11–13 were derived from the Resilience in Illness Model (RIM).^11,19 The RIM explains the relationship between the social protective factor (i.e., support from HCPs) and the other variables in the RIM (i.e., positive coping, hope, self-transcendence, resilience, and well-being). All RIM measures were used in previous studies with AYAs and have evidence of reliability and validity (Table 1). AYAs also completed a brief demographic form.

Statistical analyses

Description of qualitative variables was based on absolute and relative frequencies, whereas for quantitative variables, we used the median and interquartile range (IQR). Statistical comparisons between qualitative characteristics and gender were based on the nonparametric Fischer's exact test. The nonparametric Mann–Whitney test was used to test for differences regarding the levels of the quantitative variables between males and females.

We used exploratory factor analysis to identify the underlying latent factors of the C-HCPS. We used principal component factor extraction and selected the number of latent factors using Kaiser's criterion. Oblique rotation was applied to enhance the interpretability of the factors and to allow these factors to be associated. To better understand the association between the C-HCPS and the hypothesized scales, we generated scatterplots, including a smooth association curve that was estimated using locally weighted regression (Lowess). Reliability assessment of the C-HCPS was based on Cronbach's alpha coefficient. Convergent and discriminant validity of the C-HCPS was evaluated through the nonparametric Spearman's rank correlation coefficient (Spearman's rho). All analyses were performed using STATA 14.

Results

The initial study sample consisted of 103 participants. However, 2 participants had missing values in multiple variables and were excluded from the analysis, leading to a total of 101 patients for the current analysis. The majority of the participants were males (67.3%), non-Hispanic or Latino (89.7%), and White (89.8%). Descriptive characteristics of the study sample, both by gender and overall, are presented in Table 2.

Aim 3: Assess the dimensionality of the instrument

Our initial exploratory factor analysis results revealed that the 93 items of the C-HCPS had 16 dimensions, which did not make sense. Upon closer examination, we realized that the disconnectedness items (mixed in with the connectedness items) stood out as their own concept. Therefore, we made the decision to systemically reduce the C-HCPS items and separate the disconnectedness items into their own scale—the Disconnectedness with HCPs Scale (D-HCPS).

We reduced the number of items of the C-HCPS and D-HCPS by completing two steps. First, we used the CVI scores from both AYAs and clinicians, described in phase I, to calculate a content validity ratio (CVR).²⁰ The range for the CVR is −1 (none of the responders considered the item as essential) to 1 (all responders considered the item as essential). A value of CVR = 0 indicates that 50% of the responders rated the item as essential. Retained items had a CVR value of at least 0.672 for clinicians and 0.736 for AYAs.

Our second step involved comparing the CVR scores and evaluating items for redundancy. If two items had an adequate CVR, but one item better represented the experience over the other, it was deleted. For example, we eliminated “my doctors/nurses are good about telling me what I should know about my diagnosis and treatment” and kept “my doctors/nurses are honest with me.” Based on these two steps, 46 items were deleted (35 connectedness items and 11 disconnectedness items were retained).

We then reran the exploratory factor analysis separately for both the C-HCPS and D-HCPS. Results from the exploratory factor analysis are presented in Tables 3 and 4.

Table 3. Exploratory Factor Analysis of the Connectedness with Health Care Providers Scale

Gray shading indicates values that loaded significantly on each factor (above 0.5).

Table 4. Exploratory Factor Analysis of the Disconnectedness with Health Care Providers Scale

Gray shading indicates values that loaded significantly on each factor (above 0.5).

Five factors were identified for the C-HCPS, defined as follows: (1) Empathic, inclusive communication—skilled communication by HCPs that is understanding, thoughtful, and encourages open two-way communication. (2) Trust and gratitude for confident HCPs—a sense of trust and appreciation for HCPs who are competent in their abilities. (3) Reciprocity—a satisfying, mutual give and take relationship. (4) Tailored communication—personalized and understandable information. (5) Trustful communication—honest communication about the cancer and treatment.

Two factors were identified for the D-HCPS, defined as follows: (1) Disrespectful of personhood—behaviors that ignore and minimize the AYA as a valued person. (2) Characteristics of HCPs who foster disconnectedness—attitudes that create a sense of disconnectedness.

Aim 4: Evaluate the internal consistency reliability of the instrument

Reliability of the C-HCPS was very high, with Cronbach's alpha coefficient being 0.964 and average interitem covariance 0.174. Similarly, Cronbach's alpha coefficient for the D-HCPS was 0.881, with the average interitem covariance being 0.314.

Aim 5: Evaluate the convergent and discriminant validity of the instrument through hypothesis testing

Good evidence of convergent and discriminant validity was found for the C-HCPS. Convergent validity was exhibited by significant positive associations (p ≤ 0.05)) with seven of the eight hypothesized scales: (1) PSQ (Spearman's rank correlation coefficient: 0.493), (2) Perceived Social Support from Health Care Professionals (Spearman's rank correlation coefficient: 0.681), (3) Confrontive/Optimistic/Supportant score (Jalowiec Coping scale—Positive Coping) (Spearman's rank correlation coefficient: 0.270), (4) Herth Hope Index (Spearman's rank correlation coefficient: 0.327), (5) Reed Self-Transcendence Scale (Spearman's rank correlation coefficient: 0.214), (6) Haase Adolescent Resilience in Illness Scale (Spearman's rank correlation coefficient: 0.270), and (7) Mishel Uncertainty in Illness Scale (Spearman's rank correlation coefficient: 0.324). Discriminant validity was exhibited by nonsignificant relationships with all five hypothesized scales.

The D-HCPS also had good evidence of convergent validity. Convergent validity was exhibited by significant negative associations (p ≤ 0.05) with four of the eight scales: (1) PSQ-18 (Spearman's rank correlation coefficient: −0.61); (2) Perceived Social Support with HCPs (Spearman's rank correlation coefficient: −0.486); (3) Haase Adolescent Resilience in Illness Scale (Spearman's rank correlation coefficient: −0.222); and (4) Mishel Uncertainty in Illness Scale (Spearman's rank correlation coefficient: −0.225). Discriminant validity was demonstrated by the hypothesized nonsignificant relationships with all five scales.

The C-HCPS and D-HCPS were moderately correlated with each other (p < 0.001; Spearman's rank correlation coefficient: −0.582). Higher sense of connectedness is associated with a lower sense of disconnectedness.

Last, there was no evidence for a difference between males and females with respect to connectedness (p-value = 0.519) and disconnectedness (p-value = 0.520). Moreover, there was no evidence for an association between the connectedness scale scores and age (p-value = 0.974). However, there was a negative association between disconnectedness scale scores and age (Spearman's rho = −0.1874), which was only marginally nonsignificant (p-value = 0.068).

Discussion

Results indicate that both the C-HCPS and D-HCPS are reliable and show evidence of convergent/discriminant validity. The initial dimensionality of the C-HCPS was a surprising, but important, finding. We did not expect the disconnectedness items to influence the factor structure. When we first developed the C-HCPS, we thought disconnectedness was the polar opposite of connectedness. Thus, we expected the disconnectedness items to group together. However, upon closer examination of the findings of our original phenomenological study, we realized that disconnectedness is its own separate concept rather than the polar opposite of connectedness. Connectedness implies a continuum, ranging from well connected to unconnected (no connection at all). In contrast, disconnectedness is an immediate reaction when HCPs exhibit characteristics or behaviors that destroy the opportunity to connect. Thus, unconnectedness would be the polar opposite rather than disconnectedness.

Recommendations for future research

Further research is required to evaluate the reliability and factor structure of the C-HCPS and D-HCPS. We suggest reevaluating the scales in a larger sample, using confirmatory factor analysis. It is also worthy to note that reliability of the C-HCPS is very high. Future research is needed to help reduce the C-HCPS. In addition, confirmatory factor analysis in a different population will provide explicit evidence for the goodness of fit of the proposed factor structure.

Another recommendation for research is to examine the magnitude of connectedness and disconnectedness experiences. In our phenomenological study, we found that AYAs felt connected to their HCPs (despite having a few disconnectedness experiences). Their overall sense of connectedness influenced their willingness to engage in their care and return for long-term follow-up. However, this may not be the same for all AYAs. AYAs who have more disconnectedness experiences than connectedness experiences are likely to generally feel disconnected from their HCPs. This association could not be evaluated in this study for two reasons: (1) the initial questionnaire had both disconnectedness and connectedness items mixed in together and (2) the initial questionnaire asked AYAs to report on average how many HCPs they were thinking of when filling out the questions at the end of the questionnaire. Since the majority of the items were positive, they may have only been thinking of the HCPs they were most connected too. Thus, they answered the disconnectedness items differently than if they were answered separately.

To capture the magnitude of connectedness and disconnectedness experiences, we recommend adding a statement in the instructions asking AYAs to report how many HCPs come to mind before completing the C-HCPS and D-HCPS. For example, for the C-HCPS, provide the definition of connectedness and then ask “How many HCPs do you feel connected to?” The same statement would also be added to the D-HCPS.

Last, further research is needed to determine cutoff scores of connectedness and disconnectedness and to examine whether or not connectedness with HCPs can predict positive health outcomes of AYAs. For example, the ability to assess the meaningfulness of connectedness could improve the quality of life of AYAs during treatment.

Clinical implications

The C-HCPS can be used in clinical practice to identify AYAs who are well connected with their HCPs so that continuity can be established with whom AYAs feel connected too. Moreover, we can use the D-HCPS to identify AYAs (and aim to prevent AYAs from) feeling disconnected with their HCPs. The D-HCPS can also be used to help identify HCPs who struggle to connect with AYAs and teach them strategies that AYAs perceive as essential to fostering connectedness. These strategies are published elsewhere.¹

Strengths

This study has two strengths. First, this is the first study to develop and evaluate an instrument of connectedness and disconnectedness with HCPs for AYAs. Second, the sample size was adequate for initial testing of the instruments and there was a good representation of gender and the most common cancer types seen in the AYA population.

Limitations

A limitation was that there was a limited representation of race, implying that the findings may not be generalized to all AYAs.

Conclusions

The C-HCPS and D-HCPS can be used in clinical practice to identify the degree to which AYAs are either connected or disconnected with their HCPs, which may have important clinical implications such as AYA participation in decision-making and long-term follow-up. However, further research is needed to evaluate the factor validity of the instruments.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This article was supported by Alex's Lemonade Stand Foundation (Intermediate Nurse Research Award).