Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that supports research on pragmatic trials. It is a platform that collects and shares clean trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological studies to examine the effects of treatment across trials with different levels of pragmatism, as well as other design features.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the usage of the term "pragmatic" is not consistent and its definition and evaluation requires further clarification. Pragmatic trials should be designed to inform policy and clinical practice decisions, not to confirm a physiological or clinical hypothesis. A pragmatic trial should aim to be as close as is possible to actual clinical practices which include the recruitment of participants, setting up, delivery and execution of interventions, determining and analysis outcomes, and primary analysis. This is a major distinction from explanatory trials (as described by Schwartz and Lellouch1), which are intended to provide a more complete confirmation of an idea.
Studies that are truly pragmatic should be careful not to blind patients or healthcare professionals in order to cause bias in estimates of treatment effects. Practical trials should also aim to attract patients from a variety of health care settings, so that their results can be applied to the real world.
Furthermore the focus of pragmatic trials should be on outcomes that are vital for patients, such as quality of life or functional recovery. This is particularly relevant when it comes to trials that involve the use of invasive procedures or potential serious adverse events. The CRASH trial29 compared a 2 page report with an electronic monitoring system for patients in hospitals with chronic heart failure. The catheter trial28, however was based on symptomatic catheter-related urinary tract infection as its primary outcome.
In addition to these features pragmatic trials should reduce the trial's procedures and requirements for data collection to reduce costs. Furthermore, pragmatic trials should seek to make their results as applicable to clinical practice as possible by making sure that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Despite these requirements however, a large number of RCTs with features that challenge the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This could lead to false claims of pragmatism, and the term's use should be standardised. The development of a PRECIS-2 tool that can provide an objective, standardized evaluation of pragmatic aspects is a good start.
Methods
In a pragmatic study it is the intention to inform policy or clinical decisions by showing how an intervention can be integrated into routine treatment in real-world contexts. Explanatory trials test hypotheses about the cause-effect relationship within idealised environments. In this way, pragmatic trials could have a lower internal validity than studies that explain and be more susceptible to biases in their design analysis, conduct, and design. Despite their limitations, pragmatic studies can provide valuable data for making decisions within the healthcare context.
The PRECIS-2 tool evaluates the degree of pragmatism within an RCT by assessing it on 9 domains ranging from 1 (very explanatory) to 5 (very pragmatic). In this study, the areas of recruitment, organization, flexibility in delivery, flexible adherence, and follow-up were awarded high scores. However, the principal outcome and method of missing data were scored below the practical limit. This indicates that a trial can be designed with well-thought-out practical features, but without harming the quality of the trial.
It is, however, difficult to assess how practical a particular trial is since pragmaticity is not a definite attribute; some aspects of a trial can be more pragmatic than others. Additionally, logistical or protocol modifications during the course of an experiment can alter its pragmatism score. Koppenaal and colleagues found that 36% of 89 pragmatic studies were placebo-controlled, or conducted prior to licensing. Most were also single-center. This means that they are not very close to usual practice and are only pragmatic in the event that their sponsors are supportive of the absence of blinding in these trials.
A typical feature of pragmatic studies is that researchers try to make their findings more relevant by studying subgroups within the trial. This can result in imbalanced analyses and less statistical power. This increases the possibility of omitting or misinterpreting differences in the primary outcomes. This was a problem during the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not adjusted for covariates' differences at the baseline.
Furthermore, pragmatic studies can pose difficulties in the collection and interpretation of safety data. This is due to the fact that adverse events are generally reported by the participants themselves and are susceptible to delays in reporting, inaccuracies, or coding variations. Therefore, it is crucial to improve the quality of outcome ascertainment in these trials, and ideally by using national registries instead of relying on participants to report adverse events in the trial's database.
Results
While the definition of pragmatism does not require that all clinical trials be 100% pragmatic, there are benefits to including pragmatic components in trials. These include:
By incorporating routine patients, the results of trials can be translated more quickly into clinical practice. However, pragmatic kr can also have disadvantages. For instance, the appropriate type of heterogeneity could help a study to generalize its findings to a variety of settings and patients. However the wrong type of heterogeneity may reduce the assay's sensitivity, and thus lessen the ability of a study to detect small treatment effects.
Numerous studies have attempted to categorize pragmatic trials, with various definitions and scoring systems. Schwartz and Lellouch1 created a framework to distinguish between explanatory trials that confirm a clinical or physiological hypothesis, and pragmatic trials that aid in the selection of appropriate therapies in the real-world clinical setting. The framework was comprised of nine domains evaluated on a scale of 1-5 which indicated that 1 was more lucid while 5 was more pragmatic. The domains covered recruitment, setting up, delivery of intervention, flexible compliance and primary analysis.
The initial PRECIS tool3 had similar domains and a scale of 1 to 5. Koppenaal et al10 developed an adaptation of this assessment, dubbed the Pragmascope that was simpler to use for systematic reviews. They discovered that pragmatic reviews scored higher across all domains, however they scored lower in the primary analysis domain.
The difference in the main analysis domain could be due to the fact that the majority of pragmatic trials process their data in an intention to treat method, whereas some explanatory trials do not. The overall score was lower for systematic reviews that were pragmatic when the domains of the organization, flexibility of delivery and follow-up were merged.
It is important to remember that a pragmatic trial does not necessarily mean a low-quality trial, and in fact there is a growing number of clinical trials (as defined by MEDLINE search, however it is neither sensitive nor specific) that use the term 'pragmatic' in their abstracts or titles. The use of these words in abstracts and titles could suggest a greater awareness of the importance of pragmatism, but it is unclear whether this is manifested in the content of the articles.
Conclusions
In recent years, pragmatic trials have been becoming more popular in research as the value of real-world evidence is becoming increasingly acknowledged. They are clinical trials that are randomized which compare real-world treatment options instead of experimental treatments under development. They include patient populations which are more closely resembling the ones who are treated in routine care, they use comparisons that are commonplace in practice (e.g. existing medications), and they depend on the self-reporting of participants about outcomes. This method can help overcome the limitations of observational research for example, the biases that are associated with the reliance on volunteers and the lack of codes that vary in national registers.
Pragmatic trials have other advantages, including the ability to draw on existing data sources and a higher probability of detecting meaningful differences than traditional trials. However, they may have some limitations that limit their effectiveness and generalizability. For instance, participation rates in some trials could be lower than anticipated due to the healthy-volunteer influence and incentives to pay or compete for participants from other research studies (e.g. industry trials). The need to recruit individuals in a timely fashion also reduces the size of the sample and impact of many pragmatic trials. In addition certain pragmatic trials don't have controls to ensure that the observed differences are not due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatism. The PRECIS-2 tool was employed to determine the degree of pragmatism. It covers areas such as eligibility criteria as well as recruitment flexibility, adherence to intervention, and follow-up. They found 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or above) in at least one of these domains.
Trials with high pragmatism scores are likely to have more lenient criteria for eligibility than traditional RCTs. They also contain populations from various hospitals. These characteristics, according to the authors, could make pragmatic trials more useful and relevant to the daily clinical. However, they don't guarantee that a trial will be free of bias. Furthermore, the pragmatism of the trial is not a fixed attribute and a pragmatic trial that doesn't contain all the characteristics of an explanatory trial can produce valuable and reliable results.