Arthritis & Rheumatism
Official Journal
of the American College of Rheumatology
Vol. 38, No. 6, June 1995
Copyright © 1995, by the American College of
Rheumatology
Reprinted from Arthritis & Rheumatism
David
T. Felson, Jennifer J. Anderson, Maarten Boers, Claire Bombardier, Daniel
Furst, Charles Goldsmith, Linda M. Katz, Robert Lightfoot, Jr., Harold
Paulus, Vibeke Strand, Peter Tugwell, Michael Weinblatt, H. James Williams,
Frederick Wolfe, And Stephanie Kieszak
From
the Committee on Outcome Measures in Rheumatoid Arthritis Clinical
Trials, a subcommittee of the Committee on Health Care Research, American
College of Rheumatology.
David T. Felson, MD, MPH, Jennifer J. Anderson, PhD: Boston
University Arthritis Center, Boston, Massachusetts; Maarten Boers,
MD, PhD, MSc: University Hospital, Maastricht, The Netherlands; Claire
Bombardier, MD: Wellesley Hospital, University of Toronto, Toronto,
Ontario, Canada; Daniel Furst, MD: Virginia Mason Medical Center,
Seattle, Washington; Charles Goldsmith, PhD: McMaster University,
Hamilton, Ontario, Canada; Linda M. Katz, MD, MPH: Food and Drug Administration,
Rockville, Maryland; Robert Lightfoot, Jr., MD: University of Kentucky,
Lexington; Harold Paulus, MD: University of California at Los Angeles
Vibeke Strand, MD: Stanford University, Stanford, California; Michael
Weinblatt, MD: Brigham and Women's Hospital, Boston, Massachusetts;
H. James Williams, MD: University of Utah, Salt Lake City; Frederick
Wolfe, MD: Arthritis Center, Wichita, Kansas; Stephanie Kieszak, MA:
American College of Rheumatology, Atlanta, Georgia.
The opinions and assertions contained herein are the private
views of the authors and are not to be construed as official or as
a reflection of the views of the Food and Drug Administration.
Submitted for publication September 2, 1994; accepted
in revised form December 6, 1994.
Objective.
Trials of rheumatoid arthritis (RA) treatments report the average response
in multiple outcome measures for treated patients. It is more clinically
relevant to test whether individual patients improve with treatment,
and this identifies a single primary efficacy measure. Multiple definitions
of improvement are currently in use in different trials. The goal of
this study was to promulgate a single definition for use in RA trials.
Methods.
Using the American College of Rheumatology (ACR) core set of outcome
measures for RA trials, we tested 40 different definitions of improvement,
using a 3-step process. First, we performed a survey of rheumatologists,
using actual patient cases from trials, to evaluate which definitions
corresponded best to rheumatologists' impressions of improvement, eliminating
most candidate definitions of improvement. Second, we tested 20 remaining
definitions to determine which maximally discriminated effective treatment
from placebo treatment and also minimized placebo response rates. With
8 candidate definitions of improvement remaining, we tested to see which
were easiest to use and were best in accord with rheumatologists' impressions
of improvement.
Results.
The following definition of improvement was selected: 20% improvement
in tender and swollen joint counts and 20% improvement in 3 of the 5
remaining ACR-core set measures: patient and physician global assessments,
pain, disability, and an acutephase reactant. Additional validation
of this definition was carried out in a comparative trial, and the results
suggest that the definition is statistically powerful and does not identify
a large percentage of placebo-treated patients as being improved.
Conclusion.
We present a definition of improvement which we hope will be used widely
in RA trials.
Recent work by our
committee in concert with the international rheumatology community has
led to the development of a uniform core set of outcome measures for
rheumatoid arthritis (RA) trials (1). While this core set represents
an advance in defining and standardizing the outcomes to be measured
in RA trials, it has not changed the focus of trial reporting and analysis,
i.e., average improvement for each of the outcomes measured. Usually
clinical trials in RA report the average (mean or median) improvement
experienced by treated patients, with the average improvement with one
treatment compared with the average improvement with another.
Unfortunately, this current practice is problematic: moderate average
improvement of patients undergoing a treatment may occur because all
patients improved modestly or because half of the patients experienced
dramatic improvement and the other half no improvement at all. Further,
testing for significant results in each of 7 core set measures increases
the likelihood of detecting a difference between therapies when no real
difference exists (a Type I error) and makes it difficult to interpret
the difference between therapies when just 1 or 2 outcome measures are
significantly different (Are 2 therapies different if 1 of such outcomes
shows significant differences between treatment groups? Two of 7? etc.)
The availability of a single definition of response in RA trials would
resolve this problem. It would be a single primary end point for analysis.
Problems associated with multiple testing would diminish. If a uniform
definition of improvement were used, the percentage of patients improving
could be compared across trials, with the caveat that patients in different
trials are different and may not be equally likely to improve given
the same therapy.
Furthermore, patients are interested in the likelihood that they themselves
will improve, not in the average response of similar patients being
treated. Also, a focus on which patients improve in trials could lead
to investigations that characterize what types of patients improve with
different therapies. Current practice does not allow this, since individual
patients are not well characterized by reports of trials. Last, as will
be shown below, relying on a single definition of improvement that incorporates
information from several outcome measures can substantially enhance
the statistical power of a trial.
Existing
Definitions Of Improvement
Definitions of
improvement have been developed previously. First, the American Rheumatism
Association (now the American College of Rheumatology [ACR] defined
remission in RA (2), but remission occurs so rarely in trials that it
has not been a useful outcome measure for trials.
Using data from multicenter RA trials, Paulus et al (3) developed a
definition of improvement based on a set of measures that discriminated
well between active second-line drug treatment and placebo and that
limited placebo response to ~5%. This definition requires response
in at least 4 of 6 selected measures. These include a 20% improvement
in morning stiffness, erythrocyte sedimentation rate (ESR), joint tenderness
score, and joint swelling score and improvement by at least 2 grades
on a 5-grade scale (or from grade 2 to grade 1) for patient and physician
global assessments of current disease severity.
This definition of improvement is clinically reasonable and workable
in the context of trials, but it has been used inconsistently. Although
it was developed with statistical discrimination in mind, it may not
correspond to the patient's or clinician's perception of clinical improvement.
In addition, it relies on global severity scales that are unique to
trials from the Cooperative Systematic Studies of the Rheumatic Diseases
(a 5-point adjectival scale), and are not widely used elsewhere. The
5-point adjectival scale may not be as sensitive to change as a 7-point
scale or a 10-cm visual analog scale (4). Furthermore, elements included
in the Paulus improvement criteria do not correspond to the current
core set: morning stiffness, a measure often insensitive to change,
is included, and measurement of physical function is excluded. Joint
counts, morning stiffness, and ESR are equally weighted in the Paulus
criteria, whereas studies of clinician perception of improvement suggest
that joint counts are emphasized more heavily (5).
Dutch investigators (6) have suggested an index (the Disease Activity
Score [DAS]) to be used in evaluating improvement. This score, while
not easy to compute, has the advantage of drawing from several different
outcome measures to assess disease activity, with measures weighted
toward joint counts.
The investigators in many trials have created their own definitions
of improvement. For example, among 15 trials of RA treatments (other
than nonsteroidal antiinflammatory drugs) published in 1992 (references
available from the authors), only 6 used improvement or response criteria
and each used a different definition of improvement, with only 1 using
the Paulus criteria. This heterogeneity prevents comparisons of rates
of improvement across trials and provides a powerful argument in favor
of a standardized, widely used definition of improvement.
As part of an ACR
committee whose objective was to develop uniform standards for RA trial
measurements, we created a definition of improvement using elements
of the ACR core set. To achieve that goal, we drew on clinical impressions
of which RA patients improve, to identify what measures clinicians emphasize
in evaluating patient improvement. We combined this with a statistical
approach similar to that used by Paulus et al (3), with additional trial
data to allow comparison of a variety of improvement definitions. Our
statistical approach focused on the definition of improvement that best
discriminates between active drug-treated and placebo-treated patients.
The overall process is depicted in Figure 1.
Figure
1.[Click the thumbnail image above to view a larger version in
a separate browser window.]
Methods
Physician survey
(Figure 1, step 1). The first step was to assess how rheumatologists
decide whether a patient has improved. Survey studies (5) had suggested
that rheumatologists regard a patient as improved if the tender or swollen
joint count improves by ~20% or if other outcomes improve by a larger
percent. However, earlier studies combined data on clinicians and nonclinicians,
did not include all elements of the ACR core set, and did not necessarily
use data from real patients.
We therefore surveyed rheumatologists, using "paper" patients
selected from real clinical trials by stratified random sampling to
include a large number of survey patients near expected thresholds for
improvement (20-45% improvement in at least 3 outcomes). The 89 rheumatologists
to whom the survey was sent consisted of Outcome Measures in Rheumatoid
Arthritis Clinical Trials (OMERACT) committee members, participants,
and others chosen because of their considerable RA clinical and/or clinical
trial experience. Sixty-eight (76.4%) returned the surveys, and all
surveys returned were usable. The ages of the respondents ranged from
31 to 69 years (median 47 years), 15% were female, and the median number
of hours of patient care per week was 17, with 62% of the respondents
medical school based.
For each element of the core set (e.g., tender joint count), data at
baseline and at 6 months were provided and the percent change was noted.
We asked survey respondents whether each paper patient had improved
or not. Since the survey was also designed to evaluate patient worsening,
only 43 of 69 patients in the survey provided useful information on
improvement. The other 26 patients were substantially below expected
thresholds for improvement. As a validation of our assumptions about
which patients from the survey would provide useful data regarding improvement,
none of these latter 26 patients were designated as improved by more
than 14% of survey respondents.
Analysis of physician
survey (Figure 1, step 2). In the survey results, we focused on patients
characterized as improved by at least 80% of the surveyed rheumatologists.
We chose the cutoff of 80% because we were interested in patients whom
almost all rheumatologists would characterize as improved. We then examined
the extent to which these same patients were characterized as improved
according to various possible definitions of improvement, as shown below.
We also looked at the percent of false-positives, i.e., patients not identified
as improved by ³80% of rheumatologists
but classified as improved by the improvement definition. We decided that
all candidate definitions of
improvement with chi-square values <6 (which corresponds to P
= 0.01) or false-positive rates >25% would be excluded from further
consideration. Changing these thresholds did not change the relative performance
of improvement definitions in the survey.
Analysis of trial data (Figure 1, steps 3 and 4). Once the survey
results had eliminated some of the possible definitions of improvement,
we turned to statistical analysis of trial data. The goal was to select
the improvement definition(s) that best discriminated active second-line
drugs from placebo. We assembled a data set of 5 placebo-controlled trials
of second-line drugs, including 1 trial of gold (7) and 4 of methotrexate
(refs. 8-10 and Schmid FR et al: unpublished observations). One of these
(Schmid FR et al: unpublished observations) was a small unpublished trial,
and its exclusion does not affect analytic results. Since we wished to
choose regimens that offered as large as possible an efficacy difference
between drug and placebo, we excluded 1 auranofin arm in 1 trial, since
evidence (11,12) suggests it is relatively weak. Six of the 7 ACR core
set measures were included in these trials, but like many completed RA
trials, 4 of the 5 trials did not include an assessment of functional
status. We substituted grip strength, a measure whose change correlated
moderately (r = 0.45 with change in Arthritis Impact Measurement Scales
physical function in 1 trial [7] and r = 0.64 in another study [13]) and
which loads with functional status in factor analyses of trial data, suggesting
that it measures a similar construct (4).
The data set contained 508 patients, but 320 patients (177 active drug-treated/143
placebo-treated) remained after exclusion of patients with missing data
for at least 1 element of the core set (or for grip strength). Additional
analysis of the 1 trial with data on function suggests that the results
would likely not have changed if such data were available in all trials.
After selecting the improvement definition based on its performance in
placebo-controlled trials, we tested it in a large comparison trial data
set of methotrexate and auranofin, in which methotrexate had been shown
to be more efficacious (n = 271 patients with complete data) (12).
Table
1. [Click the thumbnail image above to view a larger version in a
separate browser window.]
In analyzing trial
data, we calculated the percentage of active drug-treated patients who
were identified as improved by each candidate improvement definition and
the percentage of placebo-treated patients who were characterized as improved
by each definition. For each improvement definition, we also evaluated
the statistical power in discriminating active drug from placebo groups.
The first stage of assessing candidate definitions entailed selecting
the most statistically powerful. Of those with roughly equal power, we
then chose the ones that identified the fewest placebo-treated patients
as improved. Because of the imprecision of estimates, we relied further
on the analysis of the comparative trial (methotrexate versus auranofin)
and attempted to be generous in our estimates of equivalence, so as not
to eliminate a definition of improvement because of insufficient data
.
Ease of use, credibility (Figure 1, step 5). From those definitions
remaining, we made our final choice. As a group of experienced trialists,
we ranked the face validity (clinical reasonableness and ease of use)
of the remaining definitions on a 1-8 scale with 8 the highest, and then
tabulated the ranks. Also, we returned to the rheumatologist survey and
ranked each definition by its kappa statistic (another measure of agreement
between the rheumatologists' impression of improvement and the definition's
classification of improvement). These 2 rankings were multiplied, and
the definition with the best score was selected.
Results
The survey. Of
the 43 "paper" survey patients that were the focus of our investigation
of improvement, 18 were thought by ³80%
of the respondents to have improved and 25 were not.
We tested 40 possible criteria for improvement (Table 1). These were selected
because they were used in trials, because they were recommended in publications,
by members of our committee, or by the international community, or because
they were variations on used or recommended definitions.
There were 7 groups of candidate improvement definitions. The first group
was derived from the Paulus criteria (3) and substituted improvement in
pain or physical disability for the Paulus criterias' improvement in morning
stiffness. This group of definitions was referred to as Paulus.
Another group of definitions of improvement required improvement in the
tender and swollen joint counts, as well as in a proportion of other core
set elements. Because of similar recent preliminary World Health Organization
recommendations developed by 1 of the authors (Dr. Furst), we designated
this group of improvement definitions as WHO.
A third group (called Equal) weighted each of the core set elements equally
and tested equal percent improvements in all core set elements. For example,
one definition was 20% improvement in 5 of 7 of the core set elements,
another 30% improvement in 5 of 7, and another 30% in 4 of 7, etc.
For the fourth group, developed from OMERACT meeting surveys (and therefore
called OMERACT), we used evidence that clinicians emphasized improvement
in joint counts and developed improvement definitions with ³20% improvement
in tender or swollen joint counts or at least 40% improvement in the other
measures (improvements in joint count not required).
Yet another group of definitions of improvement (called Joint Count) focused
only on joint count
measures defining improvement as improvement in tender and/or swollen
joint counts.
Figure
2. [Click the thumbnail image above to view a larger version in a
separate browser window.]
The sixth group evaluated
the recommended improvement definitions using the DAS (6), an index, and
tried out different cutpoints for improvement as well as a linearized
version (calculated using the linear regression estimate of log [esr]
over the interval 0-50 and of the square root of tender joint count over
the interval 15-45). There are 2 versions of the DAS: 1 using 2 joint
count measures and the ESR and the other using the same 3 measures plus
patient global assessment.
For the last group (called Index), we constructed pooled indices of improvement,
dividing the change in each outcome measure by its change standard deviation
(the latter derived from all trial patients) to create an effect size
for each outcome measure, and then averaging effect sizes. A change of
0.5 effect size units was used as the cutoff for improvement.
Of the 40 possible definitions of improvement tested, 17 met the previously
defined threshold in the survey, low false-positivity rate and high chi-square
value. These 17 definitions appear in boldface in Table 1. They include
all improvement definitions in groups I (Paulus) and 2 (WHO), and selected
definitions in each of the other groups. The WHO and Paulus groups of
definitions, those using the DAS, Index 3 (with 2 joint counts), and 1
of the joint count improvement criteria all had high chi-square values,
suggesting thatclinical perceptions of patient improvement rely heavily
on joint count improvement. Nonetheless, the tendency for the DAS and
joint count improvement definitions to have high false-positive rates
suggests that clinicians evaluate more than just joint count in characterizing
patients as being improved.
Table
2. [Click the thumbnail image above to view a larger version in a
separate browser window.]
Table
3. [Click the thumbnail image above to view a larger version in a
separate browser window.]
At least 1 improvement
definition from each group was included in the next stage of analysis,
but 2 that met the threshold were omitted because they were duplicative
(Ef is similar to Ed and Oc is similar to Oa) (see Table 1 for definitions
of criterion codes). In addition, at the request of committee members
and for completeness, S additional variations of the remaining 15 candidate
definitions (2 in the Index group [I2 and I7], 1 in the DAS group [Dd],
and 2 in the Equal group [Ec and Eg]) were evaluated in the next stage
with the anticipation that they might do well in discriminating active
drug- from placebo-treated patients, giving a total of 20. We planned
that later selection of an improvement definition would reincorporate
survey results, so that the added definitions that did not do well in
the survey would be appropriately penalized.
Figure
3. [Click the thumbnail image above to view a larger version in a
separate browser window.]
Analyzing trial
data. Using the previously described set of 5 placebo-controlled clinical
trials, we evaluated the proportion of active drug-treated patients designated
as improved and the proportion of placebo-treated patients as not improved
for each of the remaining definitions of improvement (see Figure 2). Curves
of equal power (isopower lines) are superimposed on the plot. Any 2 points
on the same isopower curve are definitions with equal discriminating power,
i.e., the trial sample sizes needed for those 2 definitions to detect
differences between active drug- and placebo-treated patients as significant
(2-tailed a = 0.05, power 80%) are the same.
In the lowest curve 64 patients per treatment group are needed, while
for the other 2 lines, sample sizes of 32 and 20 per group, respectively,
are required. For example, Equal definition Eb and DAS definition Da have
similar discrimination in these trial data, but they differ in the proportion
of placebo-treated and active drug-treated patients they identify as improved,
with Da identifying more of both placebo-treated and active drug-treated
patients as improved. The 2 candidate definitions discriminating best
between active and placebo treatments were 2 that did not perform well
in the physician survey, Index definition 17 and Equal definition Eg.
Definitions with the most power, that designated the fewest placebo-treated
patients as improved, were chosen (see Figure 2). These were Paulus definition
Pc, WHO definition Wc, Equal definitions Ea, Eb, Ec, and Eg, OMERACT definition
Oa, and Index definition 17. Most candidate definition groups remained
represented in this final list, although definitions of improvement based
solely on joint count improvement and those based on the DAS were eliminated.
These latter definitions had less power than the ones selected and were
especially likely to characterize placebo-treated patients as improved.
We then scored each of the 8 remaining candidate definitions of improvement
for face validity and multiplied the face validity score by the survey
kappa score (Table 2). This procedure identified 1 definition that clearly
scored better than the others, and this definition, WHO definition Wc,
was selected as the definition for improvement (Table 3). It should be
noted that not only did this definition do well in the survey (chi-square
18.1, no false-positives [Table 1]), but, in the analysis of trial data,
it discriminated well between placebo and active treatment and identified
few placebo-treated patients as improved.
Next, we tested this definition in another clinical trial data set, a
multicenter trial of methotrexate versus auranofin. In this trial, mean
improvements in individual measures were, in general, much greater for
methotrexate-treated patients than for patients receiving auranofin (12).
The definition selected and others like it in the WHO series performed
as well as or better than any other types of definitions in discriminating
between methotrexate and auranofin (Figure 3). As in placebo trials, joint
count- and DAS-based definitions identified as improved a large percentage
of patients who received the weaker therapy. The Equal definition and
the Paulus definitions characterized more methotrexate-treated and more
auranofin-treated patients as improved than did the definition selected.
Discussion
Based on this analysis
using several different approaches to evaluating potential definitions
of improvement in RA, we suggest that improvement for clinical trial patients
be defined as ³20% improvement in tender
and swollen joint counts and ³20% improvement
in at least 3 of the following 5 ACR core set measures: pain, patient
and physician global assessments, self-assessed physical disability, and
acutephase reactant. Our work suggests that this definition corresponds
closely to clinicians' impression of patient improvement since it emphasizes
joint counts, and furthermore, it discriminates powerfully between active
and placebo treatment, identifying few placebotreated patients as being
improved.
This definition of improvement provides a single outcome measure that
can be used in all RA trials. The definition of improvement can characterize
the response of individual patients to therapy, and using it, investigators
can profile those likely to respond to a therapy.
Our analyses suggest that this definition of improvement increases the
power of clinical trials since it draws on information from multiple different
outcome measures. Therefore, the sample size needed to demonstrate differences
between therapies may decrease, making it possible for some trials that
previously would have been considered to be underpowered to have sufficient
patients to compare treatments. For example, for the comparative trial
analyzed in Figure 3, between 20 and 32 patients per treatment group would
be required using this improvement definition (80% power, a = 0.05, 2-sided),
versus at least 80 patients per group if the trial were analyzed in the
current and traditional way, evaluating 1 of the 7 core set measures.
Ultimately, if the improvement criteria are widely used in a standardized
manner, it may be possible to rank the effcacy of different therapies
based on the percentage of patients who improve.
Since our data analysis focused on defining improvement based on the differences
between end of-trial and start-of-trial scores, we recommend that patients
be evaluated as improved or not improved based on their scores at trial's
end (or at the time they drop out) compared with entry scores.
Until now, improvement criteria have often relied on changes in joint
count to determine whether a patient has improved. Compared with more
comprehensive measures, definitions that depend only onjoint count generally
do not discriminate as well between active drug-treated and placebo-treated
patients, and usually identify more placebo-treated patients as being
improved. We hope that our definition of improvement satisfies a middle
ground in that it relies heavily on joint count improvement while incorporating
data from other measures.
There are limitations both to our approach to defining improvement and
to our definition. First, our analysis of how well improvement definitions
distinguished active drug-treated from placebo-treated patients was limited
by the absence of functional status data in our data sets. We had to rely
on grip strength instead. Analyses with smaller data sets that did contain
functional status suggest that the results would have been similar. Nonetheless,
it is essential that these improvement criteria be validated with data
sets that contain information on functional status change. In general,
validation in other prospectively measured data sets would be of great
value.
In addition, the use of one single measure to evaluate the response to
therapy in rheumatoid arthritis may be overly simplistic. Some treatments
affect joint count improvement more than improvement in acute-phase reactants,
and others do the opposite. To ignore the spectrum of improvement induced
by a particular treatment would be a mistake, and we recommend that the
change in each outcome still be reported, but that the primary outcome
for trials be improvement as reported here.
In summary, we suggest a definition for improvement in rheumatoid arthritis
that corresponds closely to rheumatologists' own impressions of patient
improvement and also discriminates between active drug- and placebo-treated
patients, which suggests that its use will enhance the statistical power
of future trials.
Acknowledgment
The
authors are indebted to members of the ACR Committee on Health Care Research
for their critical comments.
References
1. Felson DT, Anderson
JJ, Boers M, Bombardier C, Chemoff M, Fried B, Furst D, Goldsmith C, Kieszak
S, Lightfoot R, Paulus H, Tugwell P, Weinblatt M, Widmark R, Williams
HJ, Wolfe F: The American College of Rheumatology preliminary core set
of disease activity measures for rheumatoid arthritis clinical trials.
Arthritis Rheum 36:729-740, 1993
2. Pinals RS, Masi AT, Larsen RA, and the Subcommittee for Criteria of
Remission in Rheumatoid Arthritis of the American Rheumatism Association
Diagnostic and Therapeutic Criteria Committee: Preliminary criteria for
clinical remission in rheumatoid arthritis. Arthritis Rheum 24:1308-1315,
1981
3. Paulus HE, Egger MJ, Ward JR, Williams HJ, and the Cooperative Systematic
Studies of the Rheumatic Diseases Group: Analysis of improvement in individual
rheumatoid arthritis patients treated with disease-modifying antirheumatic
drugs, based on the findings in patients treated with placebo. Arthritis
Rheum 33:477-484, 1990
4. Anderson JJ, Felson DT, Meenan RF, Williams HJ: Which traditional measures
should be used in rheumatoid arthritis clinical trials? Arthritis Rheum
32:1093-1099, 1989
5. Goldsmith CH, Boers M, Bombardier C, Tugwell P: Criteria for clinically
important changes in outcomes: development, scoring and evaluation of
rheumatoid arthritis patient and trial profiles. J Rheumatol 20:561-565,
1993
6. Van der Heijde DMFM, Van't Hof MA, Van Riel PLCM, Theunisse Lam, Lubberts
EW, van Leeuwen MA, van Rijswijk MH, Van de Putte LBA: Judging disease
activity in clinical practice in rheumatoid arthritis: first step in the
development of a disease activity score. Ann Rheum Dis 49:916-920, 1990
7. Ward JR, Williams HJ, Egger MJ, Reading JC, Boyce E, Altz-Smith M,
Samuelson CO Jr, Willkens RF, Solsky MA, Hayes SP, Blocka KL, Weinstein
A, Meenan RF, Guttadauria M, Kaplan SB, Klippel J: Comparison of auranofin,
gold sodium thiomalate, and placebo in the treatment of rheumatoid arthritis:
a controlled clinical trial. Arthritis Rheum 26:1303-1315, 1983
8. Weinblatt ME, Coblyn JS, Fox DA, Fraser PA, Holdsworth DE, Glass DN,
Trentham DE: Efficacy of low-dose methotrexate in rheumatoid arthritis.
N Engl J Med 312:818-822, 1985
9. Furst D, Koehnke R, Burmeister LF, Kohler J, Cargill I: Increasing
methotrexate effect with increasing dose in the treatment of resistant
rheumatoid arthritis. J Rheumatol 16:313-320, 1989
10. Williams HJ, Wilkens RF, Samuelson CO Jr., Alarcon GS, Guttadauria
M, Yarboro C, Polisson RP, Weiner SR, Luggen ME, Billingsley LM, Dahl
SL, Egger MJ, Reading JC, Ward JR: Comparison of low-dose oral pulse methotrexate
and placebo in the treatment of rheumatoid arthritis: a controlled clinical
trial. Arthritis Rheum 28:721-730, 1985
11. Felson DT, Anderson JJ, Meenan RF: The comparative efficacy and toxicity
of second-line drugs in rheumatoid arthritis: results of two metaanalyses.
Arthritis Rheum 33:1449-1461, 1990
12. Weinblatt ME, Kaplan H, Germain BF, Merriman RC, Solomon SD, Wall
B, Anderson L, Block S, Irby R, Wolfe F, Gall E, Torretti D, Biundo J,
Small R, Coblyn J, Polisson R: Low-dose methotrexate compared with auranofin
in adult rheumatoid arthritis: a thirty-six-week, double-blind trial.
Arthritis Rheum 33:330-338,1990
13. Van der Heide A, Jacobs JWG, Van Albada-Kuipers GA, Kraaimaat FW,
Geenen R, Bijlsma JWJ: Physical disability and psychological well being
in recent onset rheumatoid arthritis. J Rheumatol 21:28-32, 1994
14. Fuchs HA, Pincus T: Reduced joint counts in controlled clinical trials
in rheumatoid arthritis. Arthritis Rheum 37:470-475, 1994
15. American College of Rheumatology Committee on Outcome Measures in Rheumatoid Arthritis Clinical Trials: Reduced joint counts in rheumatoid arthritis clinical trials. Arthritis Rheum 37:463-464, 1994