Permutation Tests in Clinical Trials (original) (raw)

Commentary on Randomization: The forgotten component of the randomized clinical trial

Statistics in Medicine

Commentary on Randomization: The forgotten component of the randomized clinical trial 1 | INTRODUCTION "We are what we eat." Raised on a soup of Kempthorne's Chapter 7 1 (which laid out the basis for inference from randomized clinical trials (RCTs)), Cochran's eminent practicality (don't let arcane theory prevent you from looking at actual data) in his text and classes, 2 and Cornfield's remarkable ability to experiment with and rethink his views, 3 I approach the analysis of RCTs in a somewhat different spirit from that expressed by the provocative paper by Rosenberger, Uschner, and Wang (RUW). 4 They and I start on the same page-we all accept the view that their Figure 2, the randomization model, presents the appropriate structure of inference from RCTs, while the frequently applied "invoked model" as depicted in the second panel of their Figure 1 does not. The randomization model views an RCT as a self-contained entity: it allows formal inference only into itself; inference onto a wider population requires a leap of biological, medical, and sociological faith (much to the consternation of many survey samplers 5). Often expressed in terms of the conflict between internal and external validity, this fundamental duality informs the interpretation of results of RCTs: to whom the results should apply requires judgment by physicians, regulators, payers, and patients. If the results were relevant only to participants in the trial, neither the private nor the public sector would spend hundreds of millions of dollars developing a drug or device-the conviction that the results are relevant outside the narrow confines of the trial population drives development of new therapies. The question on the table, then, is not which model is correct, but how actually to analyze data from RCTs. RUW argue that in earlier decades, statisticians knew randomization tests represented the correct approach to analysis, but their lack of access to fast computers forced use of likelihood-based methods. Now, RUW contend, the situation is reversed: we have easy access to high powered computers, but we statisticians have lost our way-we no longer appreciate the need for randomization tests. RUW are closer to knowing how students are being taught than I am-it may well be true that the crucial distinction between internal and external validity is no longer central to the training of statisticians (if that is correct, some aspects statistical education needs to revert back to earlier times). Where we part is concluding that we should therefore be replacing our current reliance on model-based inference with randomization tests. Back to Kempthorne's Chapter 7 which teaches that tests based on normal theory generally produce excellent approximations to true randomization tests. My own takeaway from that lesson was to fear ordinary statistical tests in RCTs only when some statistical aspect of the trial deviated markedly from the typical case. I have been involved in three trials whose properties were so atypical that I felt the need to perform randomization or permutation tests. The first was a trial of motexafin gadolinum in patients with brain cancer. 6 Because the trial was open label, those of us involved in its design were concerned that use of conventionally sized small blocks within centers would allow investigators to deduce the size of the blocks and therefore gain the potential to manipulate assignment of patients to the experimental or control arm. We therefore used an urn model to allocate assignments and, because of our uncertainty about the applicability of the theory that defended the use of model-based statistics, we performed a rerandomization test. The second case involved a study of fish oil for reduction of triglycerides (TG) among people whose levels were above 500 mg/dL. Here, the problem was the extreme skewness of TG levels. Rather than using medians or transformations, we decided to see how a simple analysis of covariance of level of TG with baseline TG as a covariate, stratified by the randomization strata, would compare to the same analysis based on a randomization distribution. The third case involved a Phase 3 trial of voretigene neparvovec, a gene therapy for patients with biallelic RPE65 mutationassociated retinal dystrophy. 7 The study, which used a novel outcome with a new scoring system, randomized only 21 participants to gene therapy and 10 to placebo. The small sample size, coupled with our uncertainty about the distributional properties of the scoring system, led us to do an actual permutation test. Because of the small sample size, we were able to enumerate all possible permutations of treatment assignments and calculate the permutation-test P-value. In all three

Randomization In Clinical Trials

Marmara Medical Journal, 2011

Özet Klinik denemelerde randomizasyonun amacı, hastaların deneme gruplarına tamamen şansa bağlı olarak atanmasını sağlamak ve "seçim yanlılığını" önlemektir. Bireyler, randomizasyon yöntemleri kullanılarak, çalışmaya katılma kriterleri bakımından birbirine benzer (homojen) olacak şekilde deneme gruplarına dengeli olarak atanabilirler. Ancak küçük örneklemlerde, gruplarda örneklem sayıları dengeli olmayabilir veya gruplardaki bireyler demografik özellikleri ve başlangıç klinik ölçümleri bakımından birbirine tamamen benzer olmayabilir. Bu çalışmada bu kısıtlılıklara çözüm getirebilecek randomizasyon yöntemleri örneklerle açıklanmaya çalışılmıştır.

A roadmap to using randomization in clinical trials

BMC Medical Research Methodology, 2021

Background Randomization is the foundation of any clinical trial involving treatment comparison. It helps mitigate selection bias, promotes similarity of treatment groups with respect to important known and unknown confounders, and contributes to the validity of statistical tests. Various restricted randomization procedures with different probabilistic structures and different statistical properties are available. The goal of this paper is to present a systematic roadmap for the choice and application of a restricted randomization procedure in a clinical trial. Methods We survey available restricted randomization procedures for sequential allocation of subjects in a randomized, comparative, parallel group clinical trial with equal (1:1) allocation. We explore statistical properties of these procedures, including balance/randomness tradeoff, type I error rate and power. We perform head-to-head comparisons of different procedures through simulation under various experimental scenarios...

Randomization: The forgotten component of the randomized clinical trial

Statistics in Medicine, 2018

The customary test for an observed difference … is based on an enumeration of the probabilities, on the initial hypothesis that two treatments do not differ in their effects, … of all the various results which would occur if the trial were repeated indefinitely with different random samples of the same size as those actually used."-Peter Armitage ("Sequential tests in prophylactic and therapeutic trials" in Quarterly Journal of Medicine, 1954;23(91):255-274). Randomization has been the hallmark of the clinical trial since Sir Bradford Hill adopted it in the 1946 streptomycin trial. An exploration of the early literature yields three rationales, ie, (i) the incorporation of randomization provides unpredictability in treatment assignments, thereby mitigating selection bias; (ii) randomization tends to ensure similarity in the treatment groups on known and unknown confounders (at least asymptotically); and (iii) the act of randomization itself provides a basis for inference when random sampling is not conducted from a population model. Of these three, rationale (iii) is often forgotten, ignored, or left untaught. Today, randomization is a rote exercise, scarcely considered in protocols or medical journal articles. Yet, the literature of the last century is rich with statistical articles on randomization methods and their consequences, authored by some of the pioneers of the biostatistics and statistics world. In this paper, we review some of this literature and describe very simple methods to rectify some of the oversight. We describe how randomization-based inference can be used for virtually any outcome of interest in a clinical trial. Special mention is made of nonstandard clinical trials situations.

Randomization-based inference and the choice of randomization procedures

Statistical Papers, 2019

In testing the significance of treatment effects in randomized clinical trials (RCTs), randomization-based inference is distinguished from population-based parametric and nonparametric inference, such as the t-test or permutation tests, taking into account three properties: preservation of type I error rate, relation of power to the randomization procedure, and flexibility in choosing the test statistic. In this paper, we revisit rationale of the properties and provide justification through simulations. We propose that the choice of randomization procedures and the analysis of RCTs can be facilitated by the application of randomization-based inference. Keywords Randomization tests • Population tests • Randomization procedure • Type I error rate • Statistical power 1 Randomization model Randomization tests have been a method of inference for analyzing data from randomized experiments since the days of Fisher (Anscombe 1948). Due to computational limitations, statisticians in the early days relied normal distribution theory to approximate randomization tests (Anscombe 1948

Review of Randomization Methodsin Clinical Trials

Clinical Trial Biostatistics and Biopharmaceutical Applications, 2014

In many respects, clinical trials can be seen as an art as well as a science, in that there is ample discretion for investigators to select research methods reflecting their own individual preferences. In fact, new research methods are developed on a fairly regular basis, not all of them improvements over existing methods. But the opposite trend also remains in effect, as researchers often follow established precedent, rather than thinking through the issues relevant to the current trial so as to come up with the research methods that are optimal in this case. These two forces pulling in opposite directions, individuality and inertia, compete in many aspects of clinical research, including the specific methods of randomization. New randomization methods are constantly proposed, while at the same time more and more researchers seem to be using the established standards of permuted blocks randomization or minimization (which, in its most extreme form, is not even true randomization at all). A comprehensive review of all randomization methods is beyond the scope of this work, but we will review these two established standards, as well as the newer (and vastly better) maximum tolerated imbalance procedures, including the big stick, Chen's procedure and the maximal procedure.

Randomisation in clinical trials

The British Journal of Psychiatry, 1999

BackgroundSeveral studies of papers published in non-psychiatric medical journals that report on randomised controlled trials (RCTs) indicate that there is inadequate reporting of the process by which randomisation is carried out.AimsTo examine the adequacy of the reporting of the procedure of randomisation in clinical trials of parallel design published in the British Journal of Psychiatry (BJP) and the American Journal of Psychiatry (AJP).MethodAll issues of the BJP and the AJP published between January 1990 and December 1998 were surveyed, and papers that reported on RCTs were examined to judge the adequacy of the reporting of the process of randomisation.ResultsWe found 183 papers which claimed to report on RCTs (73 in the BJP and 110 in the AJP). Nine (8.2%) of those in the AJP and six (8.2%) in the BJP described the technique of creating the randomisation sequence. Two (1.8%) of those in the AJP and 11 (15.1%) of those in the BJP described the mechanism of allocating treatment...

Randomization tests for multiarmed randomized clinical trials

Statistics in Medicine, 2019

We examine the use of randomization-based inference for analyzing multiarmed randomized clinical trials, including the application of conditional randomization tests to multiple comparisons. The view is taken that the linkage of the statistical test to the experimental design (randomization procedure) should be recognized. A selected collection of randomization procedures generalized to multiarmed treatment allocation is summarized, and generalizations for two randomization procedures that heretofore were designed for only two treatments are developed. We explain the process of computing the randomization test and conditional randomization test via Monte Carlo simulation, developing an efficient algorithm that makes multiple comparisons possible that would not be possible using a standard algorithm, demonstrate the preservation of type I error rate, and explore the relationship of statistical power to the randomization procedure in the presence of a time trend and outliers. We distinguish between the interpretation of the p-value in the randomization test and in the population test and verify that the randomization test can be approximated by the population test on some occasions. Data from two multiarmed clinical trials from the literature are reanalyzed to illustrate the methodology.

Introduction to a generalized method for adaptive randomization in trials

2013

Background Ideally clinical trials should use some form of randomization for allocating participants to the treatment groups under trial. As an integral part of the process of assessing the effectiveness of these treatment groups, randomization performed well can reduce, if not eliminate, some forms of bias that can be evident in non-randomized trials. Given the vast set of possible randomization methods to choose from we demonstrate a method that incorporates many of the advantages of these other methods.