The PRECIS-2 tool has good interrater reliability and modest discriminant validity

Kirsty Loudon, Merrick Zwarenstein, Frank Sullivan, Peter Donnan, Ildikó Gágyor, Hans Hobbelen, Fernando Althabe, Jerry A Krishnan, Shaun Treweek

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

OBJECTIVE: PRECIS - 2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS - 2 tool, unlike its predecessor, testing the discriminant validity and inter-rater reliability.

STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians and policymakers created PRECIS - 2 helping to ensure face and content validity. The inter-rater reliability of PREC IS - 2 was measured using 19 experienced trialists who used PRECIS - 2 to score a diverse sample of 15 RCT protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism.

RESULTS: Inter-rater reliability was generally good, with seven out of nine domains having an ICC over 0.65. Flexibility (Adherence) and Recruitment had wide confidence intervals but raters found these difficult to rate and wanted more information. Each of the nine PRECIS - 2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains.

CONCLUSION: We have assessed the validity and reliability of PRECIS - 2. An elaboration paper and website provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers and funders.

Original languageEnglish
Pages (from-to)113-121
JournalJournal of clinical epidemiology
Volume88
DOIs
Publication statusPublished - 8 Jun 2017

Keywords

  • research
  • pragmatic clinical trial
  • trial design
  • health care

Cite this

Loudon, Kirsty ; Zwarenstein, Merrick ; Sullivan, Frank ; Donnan, Peter ; Gágyor, Ildikó ; Hobbelen, Hans ; Althabe, Fernando ; Krishnan, Jerry A ; Treweek, Shaun. / The PRECIS-2 tool has good interrater reliability and modest discriminant validity. In: Journal of clinical epidemiology. 2017 ; Vol. 88. pp. 113-121.
@article{2bba8e7d3bb048aa9533d44292e16c46,
title = "The PRECIS-2 tool has good interrater reliability and modest discriminant validity",
abstract = "OBJECTIVE: PRECIS - 2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS - 2 tool, unlike its predecessor, testing the discriminant validity and inter-rater reliability.STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians and policymakers created PRECIS - 2 helping to ensure face and content validity. The inter-rater reliability of PREC IS - 2 was measured using 19 experienced trialists who used PRECIS - 2 to score a diverse sample of 15 RCT protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism.RESULTS: Inter-rater reliability was generally good, with seven out of nine domains having an ICC over 0.65. Flexibility (Adherence) and Recruitment had wide confidence intervals but raters found these difficult to rate and wanted more information. Each of the nine PRECIS - 2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains.CONCLUSION: We have assessed the validity and reliability of PRECIS - 2. An elaboration paper and website provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers and funders.",
keywords = "research, pragmatic clinical trial, trial design, health care, gezondheidssector, onderzoek, methodiek",
author = "Kirsty Loudon and Merrick Zwarenstein and Frank Sullivan and Peter Donnan and Ildik{\'o} G{\'a}gyor and Hans Hobbelen and Fernando Althabe and Krishnan, {Jerry A} and Shaun Treweek",
note = "Copyright {\circledC} 2017. Published by Elsevier Inc.",
year = "2017",
month = "6",
day = "8",
doi = "10.1016/j.jclinepi.2017.06.001",
language = "English",
volume = "88",
pages = "113--121",
journal = "Journal of clinical epidemiology",
issn = "1878-5921",
publisher = "Elsevier Science",

}

Loudon, K, Zwarenstein, M, Sullivan, F, Donnan, P, Gágyor, I, Hobbelen, H, Althabe, F, Krishnan, JA & Treweek, S 2017, 'The PRECIS-2 tool has good interrater reliability and modest discriminant validity' Journal of clinical epidemiology, vol. 88, pp. 113-121. https://doi.org/10.1016/j.jclinepi.2017.06.001

The PRECIS-2 tool has good interrater reliability and modest discriminant validity. / Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank; Donnan, Peter; Gágyor, Ildikó; Hobbelen, Hans; Althabe, Fernando; Krishnan, Jerry A; Treweek, Shaun.

In: Journal of clinical epidemiology, Vol. 88, 08.06.2017, p. 113-121.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - The PRECIS-2 tool has good interrater reliability and modest discriminant validity

AU - Loudon, Kirsty

AU - Zwarenstein, Merrick

AU - Sullivan, Frank

AU - Donnan, Peter

AU - Gágyor, Ildikó

AU - Hobbelen, Hans

AU - Althabe, Fernando

AU - Krishnan, Jerry A

AU - Treweek, Shaun

N1 - Copyright © 2017. Published by Elsevier Inc.

PY - 2017/6/8

Y1 - 2017/6/8

N2 - OBJECTIVE: PRECIS - 2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS - 2 tool, unlike its predecessor, testing the discriminant validity and inter-rater reliability.STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians and policymakers created PRECIS - 2 helping to ensure face and content validity. The inter-rater reliability of PREC IS - 2 was measured using 19 experienced trialists who used PRECIS - 2 to score a diverse sample of 15 RCT protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism.RESULTS: Inter-rater reliability was generally good, with seven out of nine domains having an ICC over 0.65. Flexibility (Adherence) and Recruitment had wide confidence intervals but raters found these difficult to rate and wanted more information. Each of the nine PRECIS - 2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains.CONCLUSION: We have assessed the validity and reliability of PRECIS - 2. An elaboration paper and website provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers and funders.

AB - OBJECTIVE: PRECIS - 2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS - 2 tool, unlike its predecessor, testing the discriminant validity and inter-rater reliability.STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians and policymakers created PRECIS - 2 helping to ensure face and content validity. The inter-rater reliability of PREC IS - 2 was measured using 19 experienced trialists who used PRECIS - 2 to score a diverse sample of 15 RCT protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism.RESULTS: Inter-rater reliability was generally good, with seven out of nine domains having an ICC over 0.65. Flexibility (Adherence) and Recruitment had wide confidence intervals but raters found these difficult to rate and wanted more information. Each of the nine PRECIS - 2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains.CONCLUSION: We have assessed the validity and reliability of PRECIS - 2. An elaboration paper and website provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers and funders.

KW - research

KW - pragmatic clinical trial

KW - trial design

KW - health care

KW - gezondheidssector

KW - onderzoek

KW - methodiek

UR - http://www.mendeley.com/research/precis2-tool-good-interrater-reliability-modest-discriminant-validity

U2 - 10.1016/j.jclinepi.2017.06.001

DO - 10.1016/j.jclinepi.2017.06.001

M3 - Article

VL - 88

SP - 113

EP - 121

JO - Journal of clinical epidemiology

JF - Journal of clinical epidemiology

SN - 1878-5921

ER -