Introduction

Chronic obstructive pulmonary disease (COPD) remains a leading cause of death worldwide¹ and is associated with a high economic and social burden.² A major goal of therapy in COPD is to improve lung function and provide symptom relief. Effective management of COPD has been shown to reduce exacerbations, reduce hospitalizations, improve health-related quality of life, and reduce mortality.³ Adherence to prescribed inhaled medications is key in the management of patients with COPD in both clinical and ambulatory settings.⁴ The consequences associated with nonadherence include increased risk of poor clinical outcomes, worsening quality of life, higher mortality rate, increased number of relapses, and increased health care expenses.^5,6 Adherence is further complicated because patients with COPD are often diagnosed with other diseases that greatly affect self-management of medications and financial burden.^5,6 It has been suggested that patient adherence is lower for medications that do not have an immediate or direct effect on symptoms.⁷ Evidence demonstrates that nonadherent patients may be more likely to adhere to therapy if they could feel an immediate symptomatic benefit.^8,9 The onset of action of a prescribed therapy may help patients adhere to their daily regimen because the patient perceives an immediate improvement from their medication.^9,10 To date, research regarding the onset of effect of prescribed inhaled medications in COPD populations is limited.

The Onset of Effect Questionnaire (OEQ) consists of a 5-item patient-reported outcomes questionnaire that was originally developed for patients with asthma to evaluate patient perceptions related to the onset of action of maintenance medication.^8,11 The OEQ items are as follows:

1. During the past week, you could tell your study medication was working;
2. During the past week, you could feel your study medication begin to work right away;
3. During the past week, you felt physical sensations shortly after taking your study medication that reassured you that it was working;
4. During the past week, your study medication worked as quickly as albuterol;
5. During the past week, you were satisfied with how quickly you felt your study medication begin to work.

Each OEQ item is evaluated using a 5-point scale, including: “strongly disagree,” “somewhat disagree,” “neither agree nor disagree,” “somewhat agree,” and “strongly agree.”⁸ Two of the OEQ items (Items 2 and 5) have demonstrated reliability and content validity in patients with asthma and support use in future clinical trials to evaluate asthma maintenance treatment.¹¹ Although content validity is reported in patients with asthma, the OEQ has not been evaluated in patients with COPD. The purpose of this study was to evaluate the qualitative and psychometric properties of the OEQ in patients with COPD.

Methods

Phase 1: Content Validity Qualitative Analysis

Specific Aims

The primary objective of the qualitative analysis was to evaluate the content validity of the 5 items of the OEQ in participants with mild to very severe COPD based on qualitative interviews.

Participants

Qualitative analysis participants were recruited from 4 U.S.-based clinical sites that identified the participants, verified eligibility, assisted with scheduling interviews, and completed a clinical information form for each participating patient. Participants included in this analysis were ≥40 years of age, had a spirometrically confirmed diagnosis of COPD, and were treated with any of the following therapies for COPD for ≥3 months prior to screening: long-acting muscarinic antagonist (LAMA), long-acting beta2-agonist (LABA) + LAMA, inhaled corticosteroid (ICS) + LABA, ICS + LABA + LAMA, or short-acting beta2-agonist (SABA) only. Any potential participants were excluded if they participated in a clinical trial with study interventions within 45 days of screening or had a diagnosis of any chronic respiratory conditions other than COPD. The analysis protocols were approved by a central institutional review board (Ethical & Independent Review Services), and all participants provided written informed consent prior to data collection procedures.

Interview Methods

One-on-one qualitative interviews were conducted in person or by telephone by trained interviewers using a semistructured interview guide and a combined approach of concept elicitation (CE) and cognitive interviewing (CI). The CE portion of the interview included open-ended questions to understand the patient’s experience of being able to feel the onset of their medication’s effects and, additionally, captured the patient-reported length of time to feel the medication working. The CI portion included completion of the COPD-OEQ, followed by an interview to assess the relevance and understandability of the COPD-OEQ items, response options, and instructions. Participants also completed a sociodemographic form, and sites completed a clinical questionnaire for purposes of sample description.

Analyses

The COPD Assessment Test™ (CAT) and medication class were used to describe the sample population. For qualitative data analyses, a content analysis approach was used to investigate the interview data (based on notes, interview transcripts, and audio recordings). These data were analyzed using a qualitative analysis software program, ATLAS.ti (Berlin, Germany), which allows for systematic assessment of the concepts and themes expressed by participants during the interview discussions.¹² A coding scheme was developed based on analysis objectives and the interview guide. For the CE portion, thematic codes were designed to capture general themes associated with the patient’s experience of being able to feel the onset of their medication effects. The CI portion featured codes designed to focus on the clarity, content, relevance, and interpretation consistency of the COPD-OEQ.

Phase 2: Psychometric Properties Quantitative Analysis

The secondary phase of this study included a 2-week observational assessment to examine the psychometric properties of the COPD-OEQ, including reproducibility, construct validity, and predictive validity of each COPD-OEQ item.

Specific Aims

The primary objective of the quantitative analysis was to examine the psychometric properties of the COPD-OEQ items, including validity (convergent and known groups) and reliability (test-retest) using a daily recall (COPD-OEQ Daily) and 7-day recall (COPD-OEQ Weekly) period. The reproducibility of COPD-OEQ Items 1 (working), 2 (working right away), and 5 (satisfied) was assessed among a group of heterogeneous but stable patients. Convergent validity was used to assess the extent to which the measure being evaluated related to other variables or to which it is expected to be related. Known-groups validity was examined by grouping participants into varying levels of disease severity and disease status, and test-retest reliability assessed the consistency of the outcomes by repeating the same assessment. The secondary objective of this analysis was to determine whether participants respond similarly to the COPD-OEQ Weekly versus Daily. Additional exploratory outcomes sought to evaluate responses to the COPD-OEQ items by medication class and to determine correlations between COPD-OEQ Weekly and Daily items.

Participants

Quantitative analysis participants were recruited from 12 clinical sites in the United States, where site investigators verified initial eligibility. Participants included in this analysis were ≥40 years of age, had a confirmed diagnosis of COPD, had a forced expiratory volume in 1 second (FEV₁)/forced vital capacity (FVC) ratio <0.70 and FEV₁ percentage predicted (FEV₁%pred) between 30% and 79% postbronchodilator medication and were treated with any of the following COPD controller medications for ≥3 months: LAMA, LABA + LAMA, ICS + LABA, or ICS + LABA + LAMA. Five participants who enrolled prior to the COVID-19 pandemic were excluded from the final analysis population because of procedural additions to the protocol after the start of the pandemic.

Study Design

This analysis included 3 clinical site visits (screening, baseline visit, and final visit). Participants were instructed to withhold their controller medication for 12 to 24 hours (depending on the prescribed interval) and to not use their rescue medication for 6 hours prior to the baseline visit. At the baseline visit, a weekly version of the COPD-OEQ was evaluated with a 7-day lookback before medications were taken or spirometry was performed. Then, spirometry was conducted before and 1 to 2 hours after participants took their maintenance inhalers, with differences recorded. Thereafter, the COPD-OEQ Daily was repeated 1 hour after maintenance inhaler use in the morning. During the baseline visit, participants also completed several questionnaires, including the CAT and the Patient Global Impression of Severity (PGIS). Prior to leaving the baseline visit, participants were provided with a peak flow meter to capture morning and evening peak expiratory flow (PEF) and were trained on how to use an electronic daily diary to capture the following assessments: COPD-OEQ Daily, and rescue medication use. Additionally, the COPD-OEQ Weekly, PGIS, and Patient Global Impression of Change (PGIC) were captured offsite on Day 8. At the final visit, participants completed the COPD-OEQ Weekly, CAT, PGIS, and PGIC.

Analyses

Descriptive statistics were used to evaluate demographic characteristics and results from the COPD-OEQ and other measures of patient-reported outcomes. COPD-OEQ reproducibility between baseline and final visits was assessed using the phi coefficient, with the chi-square test of homogeneity used for statistical significance. Construct validity (convergent and known groups) examined the relationship between COPD-OEQ and spirometry results via post dose percentage change in FEV₁ (absolute value in milliliters) after administration of controller medication. Bronchodilator responsiveness was determined by a change of >12% and >200mL in FEV₁ compared with baseline.¹³ For convergent validity, a correlation coefficient of <0.3 was considered weak, 0.3 to 0.7 indicated moderate, 0.7 to 0.9 was considered strong, and >0.9 indicated very strong association.¹⁴ Known-groups validity categorized participants by levels of disease severity, such as Global initiative for chronic Obstructive Lung Disease (GOLD) grade based on FEV₁/FVC ratio <0.70 postbronchodilator and postbronchodilator FEV₁%pred, PEF, or PGIS, with COPD-OEQ scores (perception and satisfaction) higher for those with lower severity than for those with greater severity. Test-retest reliability of the COPD-OEQ Weekly and Daily was assessed between baseline and Day 8 and between baseline and the final visit (Day 14) among participants who reported “no change,” as measured by PGIC and PGIS. Intraclass correlation (ICC) values >0.70 are generally considered acceptable for establishing test-retest reliability. For the secondary objective, the equivalence of the COPD-OEQ Weekly and Daily was evaluated by comparing the responses of the weekly with the distribution of the daily responses. The equivalence of the weekly versus daily responses was assessed via a mixed model for repeated measures, with COPD-OEQ response predicted as a function of mode, time from the final visit (Day 14), and PGIS. It is expected that the regression coefficient for mode would include 0. As an exploratory objective, COPD-OEQ responses were evaluated by medication class at baseline. Classifications included: (1) LABA + LAMA, (2) LAMA only, (3) ICS + LABA, (4) ICS + LABA + LAMA, and (5) ICS + LABA combined with ICS + LABA + LAMA. Additional exploratory analyses evaluated the responses to the COPD-OEQ items by medication class and the correlations between COPD-OEQ weekly and daily items.

Results

Phase 1: Content Validity Qualitative Analysis

Patient Characteristics

This analysis included 44 participants who were mostly female (54.5%) and White (88.6%) and had a mean age of 66.3 years (range: 47.0–82.0); the mean time since COPD diagnosis was 8 years (range: 1–24; Table 1). The symptom severity stratification, based on CAT scores, was as follows: <10 (n=3), 11 to 20 (n=17), 21 to 30 (n=20), and >30 (n=4). The medication groupings consisted of LAMA (n=5; 11%), LABA + LAMA (n=14; 32%), ICS + LABA (n=10; 23%), ICS + LABA + LAMA (n=10; 23%), and SABA alone (n=5; 11%).

Concept Elicitation

Most participants (75%) reported being able to feel their medication working, using phrases such as “breathing better/easier,” “chest feeling less tight/heavy,” and “feeling airways opening up.” The timing of when participants first felt their medication begin to work varied from <1 minute to ≥21 minutes, with most participants (62.5%) reporting that their medication started working in <21 minutes (Figure 1).

Cognitive Interviewing (CI)

Three OEQ items were determined to be relevant to the COPD population and understood as intended (OEQ Items 1, 2, and 5; Table 2). The content validity of 2 OEQ items was not supported (OEQ Items 3 and 4; Table 2). Based on findings from this qualitative analysis, OEQ Items 3 and 4 were not included in the subsequent quantitative analysis. Of note, the OEQ was designed to score each question individually, and does not result in a comprehensive score. Therefore, proceeding with 3 of 5 items did not impact the validity or reliability of the questionnaire.

Phase 2: Psychometric Properties Quantitative Analysis

A total of 97 participants were included in the quantitative analysis. Participants were mostly female (57.7%) and White (92.8%), and the mean age was 71.3 years (range: 53.0–86.0; Supplemental Table 1 in the online supplement). At baseline, most participants somewhat or strongly agreed with the COPD-OEQ Weekly assessment questions: 70.1% for OEQ Item 1, 59.8% for OEQ Item 2, and 60.8% for OEQ Item 5. For the COPD-OEQ Daily assessment, more than half of the participants somewhat or strongly agreed with OEQ Items 1, 2, and 5 for the duration of the analysis (Figure 2). Participants’ lung function assessment at baseline is shown in Table 3.

Convergent Validity

At the start of the study, it was hypothesized that the COPD-OEQ Weekly would have moderate to strong correlations with the PGIS and CAT, as well as moderate correlations with PROactive Physical Activity in COPD (C-PPAC) and FEV₁. Additionally, weak correlations were expected with morning and evening PEF and FVC. During the study, moderate correlations were observed for the COPD-OEQ Weekly with PGIS at the final visit for COPD-OEQ Item 5 ([satisfied]: –0.36; p <0.05) and with C-PPAC at the final visit for all COPD-OEQ items (Item 1 [working]: 0.30; Item 2 [working right away]: 0.32; Item 5 [satisfied]: 0.35; all p <0.05). Moderate correlations were observed for COPD-OEQ Weekly Item 1 with FEV₁ pre-dose and FVC pre-dose (Item 1 [working]: –0.32 and –0.34, respectively; both p <0.05), for COPD-OEQ Weekly Item 2 with FVC pre-dose (Item 2 [working right away]: –0.31, p <0.05), and for COPD-OEQ Weekly Items 1 and 2 with FEV₁ postdose (Item 1 [working] and Item 2 [working right away]: both –0.30, p <0.05) and FVC postdose (Item 1 [working]: –0.33; Item 2 [working right away]: –0.34, both p <0.05).

For the COPD-OEQ Daily, it was expected that there would be moderate to strong correlations with the PGIS, moderate correlations with the C-PPAC, and weak correlations with the morning and evening PEF. Overall, for the COPD-OEQ Daily, weak correlations (<0.3) were observed for the CAT (all items p >0.05), PGIS (Item 5 only, p <0.05), and C-PPAC (all items p <0.05). Some moderate correlations were observed with FEV₁ pre-dose (Item 5 [satisfied]: –0.36, p <0.05), and FVC pre-dose (Item 5 [satisfied]: –0.39, p <0.05). Additionally, moderate correlations were observed for FEV₁ postdose (Item 5 [satisfied]: –0.33, p <0.05) and FVC postdose (Item 5 [satisfied]: –0.38, p <0.05). Although minimal, moderate correlations were also detected with morning PEF change (Item 2 [working right away] Day 2: –0.34, Day 11: –0.31; Item 5 [satisfied] Day 2: –0.31, Day 8: –0.36, all p <0.05).

Known-Groups Validity

For the COPD-OEQ Weekly conducted at baseline, there were no significant differences between groups for PGIS or FEV₁, defined by a 12% improvement (Supplemental Table 2 in the online supplement). Similarly, at Day 2, COPD-OEQ Items 1, 2, and 5 for morning maximum PEF and COPD-OEQ Item 5 for evening maximum PEF were not significant between groups (P >0.05); COPD-OEQ Items 1 and 2 were able to discriminate between patient groups for weekly evening maximum PEF (P =0.04 and 0.01, respectively).

For the COPD-OEQ Daily conducted at baseline, there were no significant differences between groups for PGIS or >12% improvement in FEV₁ for OEQ Items 1 and 2. COPD-OEQ Daily Item 5 conducted at baseline for a >12% improvement in FEV₁ was able to discriminate participants in the expected direction (P=0.04). With the COPD-OEQ Daily conducted at Day 2, morning maximum PEF was not significant between known groups (OEQ Item 1: P=0.06; OEQ Item 2: P=0.07; OEQ Item 5: P=0.19). However, evening maximum PEF was significant between groups for OEQ Items 1, 2, and 5 (OEQ Item 1: P < 0.01; OEQ Item 2: P=0.01; OEQ Item 5: P=0.02) but in an unexpected order.

Test-Retest Reliability

For the COPD-OEQ Weekly measured on Day 8, P values were not significant (P=0.10–0.42) and ICC values were generally moderate, ranging from 0.54 to 0.68. Similarly, for the COPD-OEQ Daily measured on Day 8, P values were not significant (P=0.07–0.62) and ICC values were moderate (0.63–0.73). COPD-OEQ Item 5 had a high correlation (0.73).

Secondary and Exploratory Objective Findings

There were no differences between participant responses on the COPD-OEQ Weekly recall and COPD-OEQ Daily recall assessments (odds ratio, 1.11; 95% confidence interval, 0.94–1.31; P=0.21). As an exploratory objective, responses were evaluated by medication class. For COPD-OEQ Weekly, a greater proportion of participants with ICS-included medication at baseline strongly or somewhat agreed with the OEQ items (OEQ Item 1: 78.8%; OEQ Item 2: 66.7%; OEQ Item 5: 66.7%) compared with those without ICS (Supplementary Table 3 in the online supplement). Similar results were observed with the COPD-OEQ Daily (OEQ Item 1: 87.7%; OEQ Item 2: 73.8%; OEQ Item 5: 72.3%). For both the COPD-OEQ Weekly and Daily, OEQ Items 1, 2, and 5 were highly and significantly correlated with each other (all P <0.0001; data not shown).

Discussion

This study evaluated the content validity and psychometric properties of the OEQ in participants with mild to very severe COPD. The statements and responses assessed in the COPD-OEQ were the same as those measured in the Asthma-OEQ. In our qualitative analysis, the content validity of 3 OEQ items (Items 1, 2, and 5) was supported among participants with COPD, meaning these OEQ items (Items 1, 2, and 5) were found to be relevant, understood as intended, and able to be completed without difficulty. In the quantitative analysis of psychometric properties of the OEQ in participants with COPD, inconsistencies were found in the validity measures of the COPD-OEQ Weekly and Daily in this analysis population. In general, convergent validity tests resulted in weak correlations between the COPD-OEQ Weekly or COPD-OEQ Daily and CAT, PGIS, and C-PPAC. However, some questions in the COPD-OEQ Weekly had moderate correlations with FEV₁ and FVC. Similarly, the known-groups validity analysis suggested that neither the COPD-OEQ Weekly nor Daily instrument was able to discriminate subjects in the expected direction when comparing by PGIS, FEV₁, or morning or evening PEF. Test-retest reliability was generally moderate, with high ICC for OEQ Item 5.

The current results are not as striking as the strong reliability and validity seen with the OEQ in patients with asthma¹¹; however, these results may be explained by looking at the pathophysiology of COPD compared with asthma. Despite the clear similarities between COPD and asthma, these diseases feature distinct types of airway inflammation and mediators, resulting in differential response to therapy.^15,16 Asthma is characterized by more variability in airflow limitation compared with COPD and, therefore, may be more responsive to controller medications shortly after use.¹⁷ Thus, the magnitude of lung function change is higher in asthma than in COPD. Furthermore, patients with COPD may be less likely to perceive airway limitations than patients with asthma due to adaptations in physical activity levels with COPD.² Therefore, patients with COPD may be less likely to perceive a notable effect of their medication. Lower baseline lung function tended to be associated with higher satisfaction. In general, lower baseline lung function increases the likelihood that small changes will correlate with patients feeling the onset of medication effects. However, in some cases, reduction of hyperinflation and dynamic hyperinflation by bronchodilators may improve FVC as much as, or more than, FEV₁. Therefore, unlike asthma, the correlation between lung function, bronchodilator responsiveness, and COPD-OEQ responses may not be as linear. Due to these clinical and nonclinical differences between asthma and COPD,¹⁶ the trends in this analysis are consistent with the expectations for response to medication in patients with COPD compared with asthma.

Regarding similarity between daily and weekly recall, results confirmed that patients respond similarly to the COPD-OEQ items from the daily and weekly versions of the questionnaire. Comparable findings were also observed in the OEQ administered to patients with asthma.¹¹ However, given the possibility of questionnaire fatigue, the COPD-OEQ Daily may be considered redundant.

It was also observed that a greater proportion of patients using ICS/LABA-containing medications had an improved onset of effect (i.e., somewhat agreed and strongly agreed with the individual COPD-OEQ items). This could be due to the synergistic effect between corticosteroids and beta-agonist medications, in which corticosteroids enhance the bronchodilatory effect of beta-agonists, and beta-agonists boost the anti-inflammatory response of corticosteroids.¹⁸ Because patients do not all respond to the same medication, future investigations may be considered to expand the evaluation of the COPD-OEQ in subpopulations according to medication type.

Results reflecting differences between subgroup stratifications should be interpreted with caution due to small sample sizes and the recruitment of a convenience-based sample. Also, because of small sample sizes, it was not feasible to include subanalyses on patients of different races, smoking status, and comorbidities. Due to clinical study recruiting challenges, demographic diversity in this study is limited; results may not be representative of the overall patient population. For example, the mean patient age in the current study is 71.3 years; responses from younger patients may differ from the current results. Additionally, other clinical subanalyses could have been performed had the OEQ proven to be more robust. Furthermore, a placebo comparator in future studies may assess the possibility of the placebo effect on patient-reported length of time to medication effect. Future studies are warranted to examine a more diverse patient population and to determine how this questionnaire may be used in future clinical studies.

Although there is content validity support for the assessment of perceived onset of effect in patients with COPD, the quantitative analyses of the COPD-OEQ Weekly and Daily provided weak evidence that the COPD-OEQ is a fit-for-purpose, reliable, valid, and well-defined measure for use in patients with moderate to severe COPD. Consequently, this tool is not ready for use in COPD populations at this time. The content validity of COPD-OEQ Items 1, 2, and 5 was supported among patients with COPD, demonstrating that these COPD-OEQ items are relevant, understood, and able to be completed without difficulty. Future studies are necessary to determine if the COPD-OEQ is a reliable and valid tool in targeted subsets of patients with COPD when endotype-specific studies emerge. In particular, one population that should be further studied includes those who currently require an ICS medication for COPD control.

Acknowledgements

Author contributions: All authors were involved in conceptuliazation and/or design of the study. NF and HG were involved in funding acquisition. GH and DR were involved with data analysis and interpretation. All authors were involved in writing and revision prior to submission.

Medical writing support was provided by Kiley Margolis, PharmD, of Lumanity Communications, Inc., under the direction of the authors in accordance with Good Publication Practice guidelines (GPP 2022) and was funded by AstraZeneca (Wilmington, Delaware).

Data sharing statement: Data underlying the findings described in this manuscript may be obtained in accordance with AstraZeneca’s data sharing policy described at https://astrazenecagrouptrials.pharmacm.com/ST/Submission/Disclosure. Data for studies directly listed on Vivli can be requested through Vivli at www.vivli.org. Data for studies not listed on Vivli could be requested through Vivli at https://vivli.org/members/enquiries-about-studies-not-listed-on-the-vivli-platform/. AstraZeneca Vivli member page is also available outlining further details: https://vivli.org/ourmember/astrazeneca/.

Declaration of Interests

CS is a paid consultant to AstraZeneca, Bronchus, Morair, and PulManage; and received COPD research grants paid to the Medical University of South Carolina from CSA Medical, Renovion, and Nuvaira in COPD. BJM has received consulting fees from Optimum Patient Care Global Limited, Third Pole, and Verona; royalties from Wolters Kluwer Health (UpToDate); grants and contracts from the National Heart, Lung, and Blood Institute, the American Lung Association, and the Department of Defense; and reports participation on data safety monitoring boards for Baystate Medical Center, Mount Sinai, and Spiration; he serves on medical advisory boards for GlaxoSmithKline, Boehringer Ingelheim, and AstraZeneca; and has received compensation for CME activity from the American College of Chest Physicians, the Eastern Pulmonary Society, Mt. Sinai, Pri-Med, and WebMD. FJT, JME, NF, and HNG are employees of AstraZeneca. GH and DR are employees of Evidera, which was contracted by AstraZeneca for study design and to collect and analyze data for this study.