Reporting on quality of life in randomised controlled trials: bibliographic study (original) (raw)

  1. Research
  2. Reporting on quality...
  3. Reporting on quality of life in randomised controlled trials: bibliographic study

Papers BMJ 1998;317 doi: https://doi.org/10.1136/bmj.317.7167.1191 (Published 31 October 1998) Cite this as: BMJ 1998;317:1191

Loading

  1. Caroline Sanders, research assistant (Caroline.Sanders{at}bristol.ac.uk),
  2. Matthias Egger, reader in social medicine and epidemiology,
  3. Jenny Donovan, senior lecturer in health and health care,
  4. Deborah Tallon, research assistant,
  5. Stephen Frankel, professor of epidemiology and public health medicine.
  6. Department of Social Medicine, University of Bristol, Canynge Hall, Bristol BS8 2PR
  7. Correspondence to: Ms Sanders

Abstract

Objectives : To examine the frequency and quality of reporting on quality of life in randomised controlled trials.

Design : Search of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.

Main outcome measures : Prevalence of reporting on quality of life. Conditions and interventions studied in trials reporting on quality of life. Quality of reporting on quality of life.

Results : During 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%)

Conclusions : Less than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.

Key messages

Introduction

Quality of life has become an accepted end point in clinical research trials in recent years, as interest in patients' experiences and preferences has grown.1Quality of life is of particular importance in trials comparing treatments with similar or no impact on disease progression and survival. However, the term quality of life is often used vaguelyand without clear definition.2–4 This is not surprising, considering the broad nature of a concept that includes physical functioning (ability to carry out activities of daily living such as self care and walking around), psychological functioning (emotional and mental wellbeing), social functioning (relationships with others and participation in social activities), and perception of health status, pain, and overall satisfaction with life.5

The lack of a clear definition of quality of life is reflected in the many instruments that have been proposed to measure it. Generic measures (such as the sickness impact profile6 or the short form health survey SF-367)broadly assess physical, mental, and social health and can be used to compare conditions and treatments. Measures specific to illnesses can supplement generic measures or can be used independently.8Other methods include measures focusing on a single aspect such as pain or anxiety and individualised measures, in which patients themselves define and rate the most important aspects of their quality of life.9

Inadequate reporting of randomised controlled trials is common and hampers the appraisal of the validity and generalisability of results.10–13 In an attempt to remedy this, consolidated standards of reporting trials (CONSORT) have been developed.14To our knowledge, however, the quality of reporting on quality of life outcomes in trials has not been systematically assessed so far. In addition, it is unclear as to what extent the growth in attention to quality of life is reflected in an increasing number of reports from clinical trials. We addressed these questions in a bibliographic study based on the Cochrane Controlled Trials Register.

Methods

Prevalence of reporting on quality of life

We searched the Cochrane Controlled Trials Register on The Cochrane Library (issue 3, 1998)15to identify randomised controlled trials published in any language between 1980 and 1997 that reported on quality of life. We entered the keyword “random” to identify randomised controlled trials, the wild card (*) ensuring that all abstracts containing words with the stem “random” (such as randomly, randomised, randomisation) were included. We identified trials that reported on quality of life by combining MeSH and free text terms “quality of life.” For each year, we calculated the proportion of reports mentioning quality of life. We repeated the analysis for cancer trials, using the search strategy of the Cochrane Cancer Network for this purpose. We also examined cardiovascular trials: we exploded all subheadings of the MeSH term “cardiovascular-diseases” and combined the results with free text searches using a total of 28 key words (details of the search strategy are available on the_BMJ_ website).

Assessment of abstracts

We examined the abstracts of reports published from 1993 to 1996. We had identified these in a search of an earlier issue of the Cochrane Controlled Trials Register (issue 4, 1997) using the same keywords. Abstracts from articles that reported the results of randomised controlled trials and included at least one outcome that we considered to be related to quality of life were assessed by CS and checked by JD. The following information was recorded: language of article, condition studied, and type of intervention.

Assessment of full reports

A sample of reports from the database of abstracts was assessed in more detail. Using a random number generator, we selected 20% of the trials reported in English. CS and DT independently examined the full reports of these using a standardised data sheet. The data sheet covered the following items:conditions and interventions studied, primary and secondary end points, sample sizes, type and documentation of the quality of life measures used, response rates, and the presentation and direction of quality of life results. If not explicitly stated by the authors, the primary end point was defined as the one that was given prominence in the report. The results relating to quality of life and other end points were also extracted. Finally, CS and DT rated trial quality using the score described by Jadad et al.16the rate of agreement between the two observers was generally high (median 80%, range 58-100%). Discrepancies were resolved in discussions with ME.

Results

Prevalence of reporting on quality of life

The keyword “random*” identified 92 364 records, which shrank to 1939 (2.1%) when “quality of life” was added. There were 10 896 cancer trials, of which 443 (4.1%) reported on quality of life, and 18 398 cardiovascular trials, of which 393 (2.1%) reported on quality of life. The proportion of trials reporting on quality of life increased over time, from 0.63% in 1980 to 4.2% in 1997 forall trials, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials (figure).

Figure

Assessment of abstracts

We found 492 reports that were published during 1993-6, of which we excluded 128 (26%) for not being a randomised controlled trial (n=59), not reporting on quality of life (n=12), duplicate entry (n=27), and not being a journal publication (n=30). We examined the abstracts of the remaining 364 reports: the most commonly studied conditions were cancer (29%) and cardiovascular disease (26%), with other diseases occurring in less than 10% of studies (table 2). The most common interventions were drugs (65%) and models of care (11%) (table 2). Most of the reports (93%) were published in English language journals.

Table 1

Subjects studied in 364 randomised controlled trials reporting on quality of life

Table 2

Interventions studied in 364 randomised controlled trials reporting on quality of life

Assessment of full reports

We selected 67 English language reports at random and examined them in detail (references are available on the BMJ's website). They were published in 52 journals. Ten journals published more than one report, and three (Circulation, Lancet, and New England Journal of Medicine) published more than two reports. The distribution of conditions and interventions studied was similar to that in the database of abstracts, with 30% of reports describing cancer trials, 25% describing cardiovascular conditions, and 64% reporting on drug interventions.

Instruments for assessing quality of life —Forty eight of the trials used at least one of the 62 established instruments shown in the box, and most (54) quoted a reference that described the methodology. In 15 trials the authors used instruments or indicators that they had developed for their study. Most of these were questionnaires, and many used visual analogue scales. In a few cases authors constructed composite indices. One of these indices, for example, was described as “the sum of asthenia + unaided dressing + visits to the shops, etc + outside activities + number of consecutive days spent without leaving home + number of days with a body temperature of 38.5°C or less.”17In two studies it was unclear how quality of life was measured.

Quality of life instruments used in 48 randomised controlled trials reporting on quality of life

Principal end points —Quality of life was the principal end point in 23 reports. Other end points were symptoms (n=12), disease progression or survival (n=8), exercise capacity (n=5), blood pressure (n=3), pulmonary function (n=2), and 14 other end points (one study each).

_Quality of reporting on quality of life_—The response rate for quality of life end points was given in 38 of the studies, with response rates ranging from 51% to 100%. In 46 reports the authors clearly stated that the patients provided information on quality of life, whereas in the remaining articles it was unclear to what extent the information originated from patients, carers, or relatives. Complete reporting of all items and scales occurred in 31 trials. Probability values were presented in 52, but only 11 provided confidence intervals. The mean Jadad quality score was 2.68, corresponding to 54% of the maximum score of 5.

Quality of reporting on quality of life was higher in the trials with quality of life as the principal end point (table 3). These trials were significantly more likely to have used an established instrument, to state respondents, to give response rates, to report on all items and scales, and to present absolute differences. Jadad scores tended to be lower for these trials, although this did not reach significance (P=0.15).

Table 3

Indicators of quality of reporting on quality of life in 67 randomised controlled trials. Values are numbers (%) of trials unless stated otherwise

Overall, 39 articles reported significantly (P<0.05) more favourable outcomes in terms of quality of life for one or more of the experimental groups. The proportion that did so was slightly higher among the trials with quality of life as the principal end point (16 (70%) v 23 (52%), P=0.17). Sample sizes were larger in studies with quality of life as the principal end point (table 3).

Discussion

Quality of life is widely accepted as an important end point in clinical trials. 1 3 5 The prevalence and quality of reporting on quality of life in clinical research is unclear, however. We examined trends in the reporting of quality of life data in randomised controlled trials during 1980-97 and assessed the quality of such reporting in more detail for 1993-6. Based on the Cochrane Controlled Trials Register, the most comprehensive source of trials available, our results indicate that the prevalence of reporting on quality of life remains trivial. Overall, the proportion of trials reporting on quality of life increased from less than 1% in 1980 to about 4% in 1997. A higher proportion might be expected for cancer considering the nature of the disease and the fact that a number of key funding bodies recently introduced policies to promote the assessment of quality of life in randomised controlled trials.18However, even among these the proportion reporting on quality of life was less than 10% in 1997. Furthermore, these low levels of reporting of quality of life may be an overestimate as almost a quarter of papers retrieved in our abstract search were considered to be irrelevant.

One reason why relatively few trialists embark on measuring quality of life may be because of methodological difficulties. Of the 67 studies sampled for detailed examination, 48 used 62 different pre-existing instruments and a further 15 studies reported new measures, with few following the methods proposed for the development and testing of instruments. 8 19The need for both generic and condition specific instruments means that a range of measures is required, but it is implausible that some 40 different measures of generic or psychological wellbeing could be justified in 48 trials. Critical analysis of these instruments was beyond the scope of our study, but the dimensions of quality of life assessed as well as the levels of validity and reliability of the instruments varied considerably. Such variations will obstruct comparisons between studies.

Quality of reporting on quality of life

Gill and Feinstein examined 75 articles with “quality of life” in their titles.2Selection was independent of study design, and most studies were published during 1987-91. They focused on the investigators' definition of quality of life, their reasons for choosing a given instrument, and the importance given to the patients' own rating of quality of life. Gill and Feinstein showed that only 15% of investigators defined quality of life, only 36% gave reasons for their choice of instrument, and only 13% invited patients to contribute personal assessments. We focused on reporting of quality of life outcomes and found that similar problems apply in clinical trials. For example, not only were patients generally not asked to supplement the questionnaire based data with personal responses, in about 30% of studies it remained unclear whether patients had contributed any information at all. Clearly, some investigators continue to believe that health professionals can make a valid assessment of their patients' quality of life, in spite of evidence to the contrary.20

Response rates are critical for reporting quality of life, as differential non-response can introduce serious bias. The fact that response rates were unreported in almost a half of the trials must be a matter of concern. Our results indicate that selective reporting of items and scales may also be a problem. We could be assured that investigators had reported all the information collected in less than half of the articles. Selective reporting of favourable or statistically significant results may thus have occurred in a number of articles.

We used the Jadad score16to examine the quality of reporting of aspects of trial methodology that have been shown to be related to the validity of randomised controlled trials.21With a mean of 54% of the maximum quality score, our sample of randomised controlled trials is of similarly low quality when compared with randomised controlled trials published in leading English language journals or trials published in languages other than English. 1122Interestingly, the Jadad score tended to be higher among the reports with quality of life as a secondary end point. Reports focusing on quality of life were more likely to use established instruments, to make clear whether the patient's view had been sought, and to offer complete reporting of items and scores. There may, however, have been some disregard for the conventions of trial design and reporting among those primarily interested in quality of life.

Conclusions

Our comprehensive analysis of the literature shows that, although it is increasing, the reporting on quality of life end points remains uncommon in randomised controlled trials, and the quality of reporting is often poor. An initiative similar to CONSORT14that involves trialists, specialists in the measurement of quality of life, and journal editors is required to develop standards of assessing and reporting quality of life in clinical trials. Decades after the invention of the randomised controlled trial, such an initiative may finally lead to trials assessing end points that really matter to patients.

Acknowledgments

The department of social medicine at University of Bristol is part of the MRC Health Services Research Collaboration.

Contributors: ME, SF, and JD designed the protocol for the study. CS performed the literature searches under the supervision of ME. Data were extracted by CS and DT. ME and CS analysed the data. All authors helped to write the paper. ME, JD, and SF are the guarantors for the study.

Footnotes

View Abstract