MetaDTA: Diagnostic Test Accuracy Meta-Analysis v2.1.3 (August 2024)


Version 2.0 is the version as described in the paper: Patel A, Cooper NJ, Freeman SC, Sutton AJ. Graphical enhancements to summary receiver operating charcateristic plots to facilitate the analysis and reporting of meta-analysis of diagnostic test accuracy data. Research Synthesis Methods 2020, https://doi.org/10.1002/jrsm.1439.

This builds on the previous version as described in the paper: Freeman SC, Kerby CR, Patel A, Cooper NJ, Quinn T, Sutton AJ. Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA. BMC Medical Research Methodology 2019; 19: 81 which can be accessed at MetaDTA version 1.27.

If you use MetaDTA please cite these papers.


MetaBayesDTA is now available!

MetaBayesDTA is an extended, Bayesian version of MetaDTA, which allows users to conduct meta-analysis of test accuracy, with or without assuming a gold standard. Due to its user-friendliness and broad array of features, MetaBayesDTA should appeal to a wide variety of applied researchers, including those who do not have the specific expertise required to fit such models using statistical software. Furthermore, MetaBayesDTA has many features not available in other apps. For instance, for the bivariate model, one can conduct subgroup analysis and univariate meta-regression. Meanwhile, for the model which does not assume a perfect gold standard, the app can partially account for the fact that different studies in a meta-analysis often use different reference tests using meta-regression.

MetaDTA 20-minute tutorial as part of ESMARConf2023


Suzanne Freeman, Clareece Nevill, Tom Morris, Naomi Bradbury, Janion Nevill, Ryan Field, Amit Patel, Nicola Cooper, Terry Quinn, Alex Sutton

For feedback/questions about this app please contact apps@crsu.org.uk

App powered by Rshiny with statistical analyses performed using the package lme4:

https://CRAN.R-project.org/package=lme4

Codes for this app are available on GitHub: https://github.com/CRSU-Apps/MetaDTA


Download a copy of the MetaDTA User Guide here:

Download User Guide

An interactive primer on diagnostic test accuracy can be found at:

https://crsu.shinyapps.io/diagprimer/

Latest update:

v2.1.3 - August 2024

Fixed a bug which prevented the risk of bias piecharts from diplaying under the SROC plots.

v2.1.2 - June 2024

Added an error message when the model doesn't converge, and removed most of the output in this case.

In previous versions, on rare occasions the model may not have converged without any warning being displayed to the user.

v2.1.1 - February 2024

Updated funding statement and logo

A full update history of MetaDTA can be found on our GitHub repository.



Funding and Support Acknowledgement:

MetaDTA is part of the Complex Reviews Synthesis Unit (CRSU) suite of evidence synthesis apps. The development of these apps is currently funded (majority) and overseen by the Evidence Synthesis Group @ CRSU (NIHR153934). Further details of other funders and support, current and past, can be found on our GitHub page . The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

More information about the UK NIHR Complex Reviews Synthesis Unit (CRSU) can be found on our website.


THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Please select a file to upload


The file should contain at least six columns. Labelling of columns is case sensitive.

The first column should be labelled author and contain the name of the study author. The author name must be unique for each study.

The second column should be labelled year and contain the year of publication.

The third column should be labelled TP and contain the number of patients with a true positive test result.

The fourth column should be labelled FN and contain the number of patients with a false negative test result.

The fifth column should be labelled FP and contain the number of patients with a false positive test result.

The sixth column should be labelled TN and contain the number of patients with a true negative test result.


Including quality assessment data (optional)

To allow the quality assessment results from the QUADAS-2 tool to be incorporated into the plots an additional seven columns are required.

The seventh column should be labelled rob_PS , representing the risk of bias in terms of the patient selection.

The eighth column should be labelled rob_IT , representing the risk of bias in terms of the index test.

The ninth column should be labelled rob_RS , representing the risk of bias in terms of the reference standard.

The tenth column should be labelled rob_FT , representing the risk of bias in terms of the flow and timing.

The eleventh column should be labelled ac_PS , representing the applicability concerns in terms of the patient selection.

The twelfth column should be labelled ac_IT , representing the applicability concerns in terms of the index test.

The thirteenth column should be labelled ac_RS , representing the applicability concerns in terms of the reference standard.

These columns should contain the numbers 1, 2 or 3 which represent low, high or unclear risk of bias/applicability concerncs respectively.


For information about the QUADAS-2 tool and how to use it please visit:

https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/

Including covariates (optional)

If any covariates are to be added to the file they should be included as the last columns in the file. If quality assessment data is not included in the file the covariates should be entered starting at the seventh column. If quality assessment data is included in the file the covariate data should be entered starting at the fourteenth column. Multiple covariates can be entered.


The default dataset, pre-loaded on the 'Data for Analysis' tab will be used for analysis if no file is selected. The 'Data for Analysis' tab will automatically update once a file is successfully loaded.

The default datasets can be downloaded using the buttons in the sidebar and used as templates to enter your own data.


Sensitivity analysis

To ensure the correct studies are excluded from sensitivity analyses please ensure that study data rows are ordered by the 'author' column alphabetically from A to Z prior to uploading to MetaDTA (Excel can do this easily).


The default dataset uses data from a systematic review investigating the accuracy of an informant-based questionnaire, for detection of all cause dementia in adults. The dataset consists of thirteen studies assessing the use of the IQCODE (Informant Questionnaire on Cognitive Decline in the Elderly) tool for identifying adults with dementia within a secondary care setting.

The IQCODE tool contains a number of questions which are scored on a five point scale. The IQCODE tool has a number of different variants, depending on how many questions are asked. The questions are based on the performance of everyday tasks related to cognitive function. These are then rated on a scale of 1-5. The final score is an average score for each question. The IQCODE tool is only a screening tool and does not offer a definitive diagnosis of dementia.

Under the 'Select example dataset' option there are four different datasets to choose from. The default is the 'Standard' dataset, which includes the author and year of each study along with the true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN). The other options add data onto this 'Standard' dataset and highlight how datasets with quality assessment scores and/or covariates should be displayed.

With this dataset there are three different covariates. The first being the country in which each individual study was conducted. The second is the threshold used in each individual study. In this case if an individuals final score was higher than the threshold the individual was classified as having dementia and would require further diagnosis. The final covariate is labelled as 'IQCODE' and indicates which variant of the tool was used in each individual study. The variants are identified by the number of questions used in the questionnaire. There are three different variants the 16-item, 26-item and 32-item.

Meta-Analysis of Diagnostic Test Accuracy Studies


Note: If the presence of zeros for two of TP, FN, FP and TN causes sensitivity or specificity to be calculated as 0/0 than an error message will appear.


Download Table

Note: Arrows to the right of the column headings can be used to sort data into ascending or descending order.

N is the total number of individuals in each study ( N = TP + FN + FP + TN )

Sens is the sensitivity, which is the probability of a positive test result given that the patient has the disease ( Sens = TP / [TP + FN] )

Spec is the specificity, which is the probability of a negative test result given that the patient does not have the disease ( Spec = TN / [TN + FP] )

Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.


Note: At least one box under 'Options for SROC plot tab' must be selected to avoid an error message


Download Plot

Click the middle of the data points for individual study summaries (an error message may occur if not selecting the middle of the pie chart when displaying risk of bias or acceptability concerns)

Note: If quality asessment data is being used and pie charts are being plotted then study weight and covariates cannot be displayed. However, if selected on the sidebar the information will still be displayed when individual studies are clicked on.

No output available - model did not converge.

No output available - model did not converge.

Below are the parameter estimates for the bivariate normal distribution for mean sensitivity and specificty (on the logit scale). Users may find these useful for further modelling e.g. inclusion of test accuracy in a decision modelling framework.


where:
Download Table

No output available - model did not converge.

Below are the parameter values required by Cochrane's RevMan software to construct plots in the ROC space for users who wish to include the analysis results as part of a Cochrane review.
Download Table

No output available - model did not converge.

No output available - model did not converge.

Sensitivity Analysis


Download Table

Note: This table only includes studies selected in the sidebar.

N is the total number of individuals in each study ( N = TP + FN + FP + TN ).

Sens is the sensitivity, which is the probability of a positive test result given that the patient has the disease ( Sens = TP / [TP + FN] ).

Spec is the specificity, which is the probability of a negative test result given that the patient does not have the disease ( Spec = TN / [TN + FP] ).

Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.


Note: At least two studies must be selected for inclusion to avoid an error message


Download Plot

Click the middle of the data points for individual study summaries (an error message may occur if not selecting the middle of the pie chart when displaying risk of bias or acceptability concerns)

No output available - model did not converge.

All studies


Selected studies only

Download Table

No output available - model did not converge.

Below are the parameter estimates for the bivariate normal distribution for mean sensitivity and specificty (on the logit scale). Estimates here are taken from the senstivity analysis model which only includes studies selected in the sidebar. Users may find these useful for further modelling e.g. inclusion of test accuracy in a decision modelling framework.


where:
Download Table

No output available - model did not converge.

Below are the parameter values required by Cochrane's RevMan software to construct plots in the ROC space for users who wish to include the analysis results as part of a Cochrane review.
Download Table

No output available - model did not converge.

Download Sensitivity Forest Plot Download Specificity Forest Plot

Note: These plots only include studies selected in the sidebar.

No output available - model did not converge.

Prevalence

Download Plot

Note: The numbers in brackets represent 95% confidence intervals.

Download Plot

Note: The sensitvity analysis tab should be visited in order to produce a plot. The numbers here will be the same as those in the meta-analysis tree diagram if no studies are excluded from the analysis in the sensitivity analysis tab.

The numbers in brackets represent 95% confidence intervals.