A reflection On Coverage Of Outcomes In The PACE Trial Compared To COS-AECOPD

By Dr. David Gillespie and Prof. Kerenza Hood

Overview

In 2014, the National Institute for Health Research (NIHR) funded a randomised controlled trial (RCT) evaluating the use of a C-reactive Protein (CRP) point-of-care test (POCT) for the management of people experiencing an acute exacerbation of chronic obstructive pulmonary disease (AECOPD) in primary care (the PACE trial). We were co-investigators on this study, and links to the protocol and several publications arising from it can be viewed at the bottom of this post.

In 2022, the European Respiratory Society (ERS) published the core outcome set (COS) for clinical trials evaluating the management of COPD exacerbations (COS-AECOPD). This work was led by Mathioudakis and colleagues, and the reference to their article is at the end of this post.

In this blog post, we aim to compare the outcomes used in the PACE trial, the extent to which they map onto the core outcome set which was developed, and reflect on both the missed opportunities for further insights during the PACE trial and the appropriateness of the developed COS for this type of research question.

Chronic obstructive pulmonary disease

COPD is a respiratory condition which affects around 2% of the UK population and is the third leading cause of death globally. People living with COPD experience breathing difficulties which gradually get worse over time, with acute exacerbations (rapid and sustained worsening of symptoms which differ from normal day-to-day variation) severely impacting on health status and quality of life.

Core outcome sets

According to the COMET Initiative, “A core outcome set (COS) is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific areas of health or health care.” The use of a COS allows studies to be compared and synthesised, and this should lead to faster and better decision making as it reduces uncertainty in evidence due to differences in outcome measurement.

The PACE trial

Our trial aimed to investigate whether adding a CRP POCT (and guidance for interpreting test results) to the management strategy for patients presenting in primary care with an AECOPD could lead to a reduction in antibiotic use without negatively impacting on subsequent COPD health status. We were mainly interested in outcomes over the four-weeks following the initial consultation, but also followed participants up at six-months for longer term outcomes. Our rationale for this study was that acute exacerbations are often managed in primary care with antibiotics, despite these typically being associated with a lower risk of hospitalisation and mortality in this setting.

COS-AECOPD

The COS-AECOPD project involved the formation of a taskforce with the following remit:

1. To develop a COS for clinical trials evaluating the management of COPD exacerbations
2. To prioritise a single instrument for measuring each of the core outcomes

In the Table below, we provide a side-by-side comparison of the outcomes specified in the COS-AECOPD and the outcomes used in the PACE trial. We reflect on each of the outcome domains within their respective sections:

Table: Comparison of COS-AECOPD and outcomes used in the PACE trial

Death

In the COS-AECOPD paper, it is noted that death is rarely evaluated in COPD exacerbation trials. This is surprising, given the 12% 90-day mortality rate within index admission for an exacerbation of COPD in the UK, as reported in the most recently published NACAP clinical audit of COPD patient outcomes. A key reason given for this is the difficulties encountered in reliability attributing cause of death. The COS group therefore opted for two death outcomes – one attributed to any cause and one related to the COPD exacerbation. For the latter, the consensus was that the primary cause of death on the death certificate should be used.

In the PACE trial, death was not part of the outcome package as while it is conceivable that inappropriately withholding antibiotics from a person experiencing an AECOPD could increase their risk of dying, this event was expected to be extremely rare in a primary care setting. Indeed, 2/649 (0.3%) randomised participants died during the four-weeks post-randomisation and both were allocated to the usual care arm.

Treatment success

The COS-AECOPD acknowledge both the difficulty and subjectivity in defining treatment success following an exacerbation, describing concerns around the variation in during of exacerbation in addition to the impact that exacerbations may have on disease progression (thereby shifting an individual’s “baseline”). These factors may drive variation in definitions of treatment success and it’s antonym, treatment failure (see Neill et al for a systematic review highlighting the wide variation in treatment failure definitions across common infections). The consensus by the panel was again to settle on a pragmatic definition which focussed on the improvement in symptoms and signs to an extent where no additional systemic treatments were prescribed.

In the PACE trial, a similar focus was taken at the trial-level. As the intervention aimed to reduce antibiotic use without adversely impacting COPD health status two weeks after consultation, the goal was that antibiotic treatment was withheld in people for whom it was not indicated and prescribed in cases where it was. Thus, the intervention would be deemed successful in the event that antibiotic use was lower and COPD health status no worse. These were classified as co-primary outcomes, and the PACE trials were one of the first antimicrobial stewardship trials to formally specify co-primary outcomes in this manner. See Gillespie et al for a brief commentary on this design. COPD health status was assessed over the first four-weeks following randomisation using the COPD Clinical Questionnaire (CCQ, see Zhou et al for details of its psychometric properties), and in addition to the initial prescribing decisions, antibiotic use and the need for further treatment were used as markers of intervention success. This is more challenging to define at an individual-level. However, one operationalisation of “treatment success” at the individual level is the absence of additional antibiotic prescription(s) following the initial consultation, which we would assume would be triggered by a treatment strategy at the initial consultation which was not associated with a satisfactory improvement in symptoms.

Need for higher level care

Key outcomes identified within this domain included the need for hospitalisation and the need for admission to the intensive care unit (ICU). However, the panel identified heterogeneity in definitions of these outcomes. For example, there is variation in the indication for hospital and ICU admission; some patients are monitored more intensively at home in situations which would require admission for others, some hospitals require ICU admission for non-invasive ventilation whereas others deliver this on respiratory wards or high-dependency units. The need for pragmatism was therefore also important for these outcomes.

In PACE, consultations with secondary care was included as a secondary outcome and defined using clinic notes reviews. This encompassed any contacts with out of hours, accident and emergency, and hospitalisations and this outcome was measured over the first four-weeks and first six-months following randomisation. Similar to death, ICU admission was not expected to be frequently reported given the setting in which participants were initially consulting.

Levels of oxygen and carbon dioxide in the blood

The inclusion of arterial blood gases as a core outcome was surprising, given the panel also indicate that this is a setting- and intervention-specific outcome and that its value in other settings (e.g. outpatient clinics) may be limited.

This was not an outcome measured in PACE. While participants returned to their general practice at four-weeks post-randomisation, this was not a measured considered relevant.

Patient-reported outcomes

These outcomes included breathlessness, health-related quality of life (HRQoL), activities of daily living, and worsening of symptoms after the initial treatment.

The panel recommended the modified Borg scale as the most appropriate way to measure breathlessness in a clinical trial of people experiencing an AECOPD. They specified that it can be self-completed and should be measured approximately the same time every day, presumably to enable valid modelling of breathlessness trajectories over time.

HRQoL-type measures have been given considerable attention in the literature, and many exist. The panel reflected on a systematic review that used COSMIN methodology to evaluate measurement properties of 23 instruments aiming to asses quality of life in COPD, which recommended the use of the COPD Assessment Test (CAT), Chronic Respiratory Questionnaire (CRQ), St Geroge’s Respiratory Questionnaire (SGRQ), or the Living With Chronic Obstructive Pulmonary Disease Questionnaire. The panel settled on the CAT, given its ease of completion and similar measurement properties compared  to the others. Both the modified Borg scale and the CAT can also be used to assess the worsening of symptoms after initial treatment.

Activities of daily living (ADL) is an outcome less frequently evaluated in trials of AECOPD. The panel reflected on methodological systematic reviews conducted to evaluate the psychometric properties of instruments in this area. They settled on the Capacity of Daily Living During the Morning (CDLM) questionnaire as an approach for measuring basic ADL (simple activities deemed essential for independent life) during an exacerbation or the Manchester Respiratory Activities of Daily Living Questionnaire (MRADL), which captured both basic and instrumental (complex activities requiring higher functioning, such as preparing meals, home maintenance, shopping, etc.) ADL at recovery from an AECOPD.

Across all of these domains, a key point noted was that it would be beneficial to obtain a “baseline” measure for these (that is, pre-exacerbation). This could be retrospectively captured at the initial study visit, and it was thought that given the acute nature of these exacerbations recall bias should be low.

In the PACE trial, patient-reported outcomes included the COPD Clinical Questionnaire (CCQ) to measure COPD health status across three domains (symptoms, function, and mental), the five-level version of the EuroQol instrument (EQ-5D-5L) to measure health utility and generic QoL, and the Chronic Respiratory Questionnaire (self-assessed, standardised version) to measure disease-specific HRQoL. These domains were therefore mostly covered (with the exception of ADL), but the recommended measures were not used. No pre-exacerbation baseline assessment was made, and thus formally defining recovery from the exacerbation was not possible.

Future impact

The panel acknowledged that measuring the future impact of an exacerbation is challenging within a trial setting, as a baseline (pre-exacerbation state) is required. Suggestions were made that the trial should be designed to include individuals during a stable period. Arguably, with a move towards data-enabled trials and the increasing use of self-monitoring apps linked to NHS services, such a design may not be necessary provided that historical routinely collected data can be linked with trial data. The panel recommended that forced expiratory volume in 1s (FEV₁) could be used to measure disease progression, provided that a baseline measure (pre-exacerbation state) can be obtained and also depending on the length of the follow-up for the study. Non-physiological measures (e.g. future exacerbations and hospitalisations) were also noted.

Future impact was not measured during the PACE trial. As mentioned above, pre-exacerbation measures were not obtained in a systematic way (primary care notes were available for participants but FEV₁ was not always recorded and it was rarely clear whether this measure was obtained during a period of stability or exacerbation). Follow-up for the trial ended at six-months post-randomisation, and so PACE might be considered to have too short a follow-up to consider the effects of the trialled intervention on “future impact” as defined in the COS-AECOPD article.

Safety

Like all trials, the panel acknowledged that serious adverse events would need capturing and reporting. Those potentially related to treatment(s) or intervention(s) are particularly important, and trial teams are encouraged to consider potential reactions related to trial participation.

Further safety aspects recommended by the panel included the development of (antimicrobial) resistant bacteria and the development of pneumonia. For the development of antimicrobial resistant bacteria, the panel’s recommendation aligned with approaches commonly used when testing antibiotic strategies in this population; that is, to focus on bacterial growth and resistance properties within spontaneously produced sputum and to consider the absence of sputum as a sign of bacterial eradication. Sputum induction was considered, but the panel encouraged researchers to consider the benefit of this against patient discomfort. Resistance to antibiotics was recommended as an outcome at baseline and within one-week of treatment completion. For the development of pneumonia, the panel recommended that this was based on imaging of the chest (e.g. chest x-ray or other imaging modalities). In hospital settings, it was also recommended that a baseline image be taken to assess presence of pneumonia.

In addition to SAE and antibiotic side effect measurement, in PACE, bacterial outcomes included the presence of potentially pathogenic bacteria in spontaneously produced sputum at four-weeks post-randomisation (available for 56% of participants), with missing sputum samples not analysed rather than not considered as coming from a participant who did not have potentially pathogenic bacteria. Resistance outcomes included resistance of the potentially pathogenic bacteria to at least one of the tested antimicrobials and resistance to selected antimicrobials of commensal organisms cultured from throat swabs. Throat swabs were chosen due to the ability to obtain these from a greater number of study participants and the importance of studying the impact of the CRP-POCT management strategy within the wider microbial community.

PACE was a trial of an antimicrobial stewardship intervention, and the measurement and analysis of microbiological outcomes in these sorts of trials are an important but often overlooked aspect – as reported by Lau et al.

Pneumonia diagnosis at four-weeks and six-months following randomisation was also assessed.

Treatment adherence

The panel recommended that an intervention-specific treatment adherence measure be reported in exacerbation trials.

In PACE, the intervention was CRP-POCT with guidance on its use. Therefore, “treatment adherence” in this context was whether the CRP-POCT was used for participants who were allocated to the intervention arm (and vice versa). Given the guidance given on cut-offs could be over-ruled based on clinical judgement, prescribing an antibiotic to someone with a low CRP value would not be considered as “not adhering” to the intervention. Nevertheless, both aspects were reported as part of the trial and sensitivity analyses were conducted adjusting for the small % of participants who did not receive their allocated intervention.

Final thoughts

There was reasonable congruence between the outcome domains used as part of the PACE trial and those recommended as part of COS-AECOPD. Where domains were missing or specific considerations not made, these were often related to the setting within which the trial was conducted (primary care). Most exacerbation trials are conducted in secondary care settings, and this is also reflected in the make-up of the respondents to the Delphi survey as part of COS-AECOPD.

A useful aspect of the COS work is the recommendations around specific measures, where they exist. Indeed, while PACE often covered a lot of the key domains – there was a distinct lack of overlap between the recommended patient-reported outcome measures in the PACE trial and those recommended by the panel. ADL is a key domain not covered in the PACE trial, and one which might be considered important to measure both from a patient perspective but also in order to better capture the cost-effectiveness of the CRP-POCT (e.g. by taking the personal social services (PSS) perspective in addition to the NHS perspective). However, the addition of this measure must be weighed against the lower severity of a primary care population and the time available to complete measures while attending general practice for an acute condition.

Table 5 in the paper maps the outcome domains to recommended instruments, and importantly these are colour coded to indicate whether the recommendation is a strong one or whether it is an interim recommended and an area that is subject to further research. If we were to design PACE now we would look at these measures first in deciding what to use. However it is important that as a trials community we do not take all of these recommendations as rote, and instead continue to develop the evidence base with a view to improving the quality of outcome reporting in these trials.

References

Bates J, Francis NA, White P, et al. General practitioner use of a C-reactive protein point-of-care test to help target antibiotic prescribing in patients with acute exacerbations of chronic obstructive pulmonary disease (the PACE study): study protocol for a randomised controlled trial. Trials. 2017 Dec;18(1):1-5.

Butler CC, Gillespie D, White P, et al. C-reactive protein testing to guide antibiotic prescribing for COPD exacerbations. New England Journal of Medicine. 2019 Jul 11;381(2):111-20.

Gillespie D, Francis NA, Carrol ED, Thomas-Jones E, Butler CC, Hood K. Use of co-primary outcomes for trials of antimicrobial stewardship interventions. The Lancet Infectious Diseases. 2018 Jun 1;18(6):595-7.

Gillespie D, Butler CC, Bates J, et al. Associations with antibiotic prescribing for acute exacerbation of COPD in primary care: secondary analysis of a randomised controlled trial. British Journal of General Practice. 2021 Apr 1;71(705):e266-72.

Gillespie D, Francis N, Ahmed H, et al. Associations with post-consultation health-status in primary care managed acute exacerbation of COPD. International Journal of Chronic Obstructive Pulmonary Disease. 2022;17:383.

Francis NA, Gillespie D, White P, et al. C-reactive protein point-of-care testing for safely reducing antibiotics for acute exacerbations of chronic obstructive pulmonary disease: the PACE RCT. Health Technol Assess 2020;24(15).

Francis NA, Gillespie D, Wootton M, et al. Clinical features and c-reactive protein as predictors of bacterial exacerbations of COPD. International Journal of Chronic Obstructive Pulmonary Disease. 2020;15:3147.

Lau TM, Daniel R, Hughes K, Wootton M, Hood K, Gillespie D. A systematic review investigating the use of microbiology outcome measures in randomized controlled trials evaluating antimicrobial stewardship interventions published between 2011 and 2021. JAC-antimicrobial resistance. 2022 Feb;4(1):dlac013.

Mathioudakis AG, Abroug F, Agusti A, et al. ERS statement: a core outcome set for clinical trials evaluating the management of COPD exacerbations. European Respiratory Journal. 2022 May 1;59(5).

Neill R, Gillespie D, Ahmed H. Variation in Antibiotic Treatment Failure Outcome Definitions in Randomised Trials and Observational Studies of Antibiotic Prescribing Strategies: A Systematic Review and Narrative Synthesis. Antibiotics. 2022 May 6;11(5):627.

Phillips R, Stanton H, Singh-Mehta A, et al. C-reactive protein-guided antibiotic prescribing for COPD exacerbations: a qualitative evaluation. British Journal of General Practice. 2020 Jul 1;70(696):e505-13.

Zhou Z, Zhou A, Zhao Y, Chen P. Evaluating the clinical COPD questionnaire: a systematic review. Respirology. 2017 Feb;22(2):251-62.

The Centre for Trials Research is funded through Welsh government by Health and Care Research Wales, and Cancer Research UK.